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Abstract 

The  inventory  routing  problem  examines  the  coordination  of  inventory  manage¬ 
ment  and  transportation  policies  when  implementing  vendor  managed  inventory  re¬ 
plenishment.  Vendor  managed  replenishment  is  the  business  practice  in  which  a 
vendor  monitors  the  inventory  levels  of  its  customers  and  decides  when  and  how 
much  inventory  to  replenish  at  each  customer  location.  The  United  States  Army 
uses  vendor  managed  inventory  replenishment  in  combat  situations  to  manage  sup¬ 
ply  operations  to  lower-echelon  organizations.  The  military  variant  of  the  stochastic 
inventory  routing  problem  requires  consideration  of  delivery  failures  due  to  hostile 
actions  taken  by  non-friendly  forces.  The  loss  of  delivery  vehicles  negatively  impacts 
future  resupply  capability.  We  formulate  a  Markov  decision  process  model  of  the 
military  inventory  routing  problem,  the  objective  of  which  is  to  determine  an  optimal 
unmanned  tactical  airlift  policy  for  the  resupply  of  brigade  combat  team  elements  in 
an  Afghanistan-like  combat  situation  using  cargo  unmanned  aerial  systems  for  deliv¬ 
ery.  Computational  results  are  presented  for  the  military  inventory  routing  problem 
with  direct  deliveries;  the  computational  example  is  based  upon  United  States  Army 
resupply  situations  that  occur  in  Afghanistan  deployments.  Base  case  results  indicate 
that  unmanned  aerial  systems  have  the  capability  to  successfully  perform  the  brigade 
combat  team  resupply  mission,  depending  on  the  dynamics  of  the  threat  situation. 
An  experimental  design  is  employed  to  determine  the  set  of  factors  important  in  a 
more  general  context. 

Key  words:  logistics,  dynamic  programming,  Markov  decision  process,  stochastic 
inventory  routing,  unmanned  aerial  vehicles 
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THE  MILITARY  INVENTORY  ROUTING  PROBLEM 


WITH  DIRECT  DELIVERY 

I.  Introduction 

The  inventory  routing  problem  (IRP)  and  associated  vehicle  routing  problem 
(VRP)  are  foundational  problems  within  the  construct  of  any  problem  where  products 
must  move  between  a  set  of  locations.  Typical  IRP  applications  model  the  supply  of 
commodities  from  a  supplier  to  a  set  of  customers.  Formulations  include  single  and 
multiple  suppliers  as  well  as  single  and  multiple  customers.  Models  range  from  ship¬ 
ping  a  single  commodity  to  multiple  commodities.  A  subproblem  of  the  IRP  is  the 
VRP,  a  variant  that  models  the  mode  in  which  the  inventory  moves  from  the  supplier 
to  a  customer.  The  type,  capacity,  and  number  of  vehicles  available  for  transporting 
inventory  are  modeled  in  the  VRP.  Moreover,  vehicles  have  constraints  that  typically 
restrict  the  distance  traveled,  time  traveled,  and  number  of  customer  allowable  in  a 
route. 

The  objective  of  both  the  IRP  and  VRP  is  usually  a  minimization  of  costs  but 
can  also  appear  as  a  maximization  of  revenue.  The  costs  considered  include  shipping 
costs  to  the  supplier  and  the  customers,  depending  on  the  particular  application. 
Holding  costs  may  apply  for  both  the  supplier  and  customers  as  well.  Depending 
on  the  motivating  problem,  the  supplier  may  not  have  enough  inventory  to  meet 
the  demand  of  the  customers,  in  which  case  a  penalty  function  also  appears  in  the 
formulation  of  the  objective.  The  shortage  may  or  may  not  be  backlogged,  causing  a 
possible  loss  of  revenue.  The  cost  of  shipping  is  also  considered,  usually  determined 
at  a  per  mile,  trip,  or  leg  basis,  thus  allowing  for  both  fixed  and  variable  shipping 
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costs.  For  a  heterogeneous  vehicle  fleet,  different  shipping  costs  apply  depending  on 
the  vehicle  type.  These  costs  can  account  for  fuel,  maintenance,  and  drivers.  In  a 
homogeneous  vehicle  fleet  formulation  these  costs  are  constant  across  the  vehicle  fleet. 

Customer  demand  in  the  IRP  may  be  either  deterministic  or  stochastic.  For 
deterministic  demand,  knowledge  of  the  demand  may  be  known  a  priori  or  revealed 
at  the  time  the  vehicle  arrives  to  the  customer.  If  the  knowledge  of  demand  quantity 
is  known  a  priori  then  shortages  are  known  at  the  onset  of  the  problem.  However,  in 
the  case  when  demand  is  revealed  upon  arrival  at  the  customer  a  shortage  may  occur 
for  that  delivery.  Depending  on  the  application,  different  formulations  address  the 
action  space  available  in  this  occurrence.  For  stochastic  demand,  the  distribution  of 
customer  demand  for  the  commodity  is  assumed  to  be  known  to  the  supplier,  although 
the  actual  demand  of  the  customer  is  not  realized  until  after  the  vehicle  arrives  to 
deliver  the  commodity,  possibly  resulting  in  shortages.  The  method  of  addressing 
shortages  at  this  point  is  the  same  as  with  the  deterministic  demand  models. 

The  rest  of  this  paper  is  organized  in  the  following  manner.  We  complete  this 
chapter  with  a  review  of  current  formulations  and  problem  classes  within  the  IRP 
and  VRP  in  the  literature.  A  focused  review  of  the  vendor  managed  inventory  replen¬ 
ishment  (VMI)  business  practice  appears  in  Section  1.1.2,  and  the  section  concludes 
with  a  description  of  the  Military  Inventory  Routing  Problem  (MILIRP).  Next,  Chap¬ 
ter  II  provides  a  general  background  for  our  problem  of  interest  and  concludes  with 
a  basic  problem  description  and  motivating  problem.  In  Chapter  III  we  formulate 
our  particular  problem  and  provide  a  discussion  of  how  we  discretized  the  continu¬ 
ous  region  considered  for  our  application,  utilize  the  threat  map  to  construct  the  arc 
lengths  between  the  nodes,  and  solve  the  VRP  subproblcm.  We  develop  our  dynamic 
programming  formulation  in  Chapter  IV  and  solve  a  small  instance  of  the  MILIRP 
exactly.  Our  results  and  conclusions  are  presented  in  Chapter  V. 
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1.1  Literature  Review 


A  review  of  current  literature  concerning  the  IRP  and  VRP  informs  our  develo- 
plent  of  the  MILIRP.  Attention  is  given  to  find  solutions  that  had  motivating  problems 
in  industries  that  behave  similarly  to  the  military.  In  particular,  we  research  any  for¬ 
mulations  that  dealt  with  vehicle  breakdown  or  failure  to  complete  a  route.  General 
purpose  solution  methodologies  and  algorithms  are  also  explored  to  gather  insight 
into  good  solution  techniques. 

1.1.1  Inventory  and  Vehicle  Routing  Formulations. 

Chien  et  al.  [2]  formulate  the  IRP  and  VRP  such  that  the  supplier  does  not 
necessarily  have  enough  inventory  to  meet  the  demand  of  all  the  customers.  The 
problem  has  a  single  supplier  node.  Profit  margins  for  each  customer  are  utilized 
to  decide  the  amount  of  inventory  delivered  to  each  customer.  Vehicle  routes  are 
constructed  so  as  to  not  limit  the  number  of  potential  customers  visited.  Upper  and 
lower  bounds  on  the  objective  function  value  are  determined  by  respectively  solving 
the  Lagrangian  relaxation  and  employing  a  heuristic  method.  Both  the  VRP  and 
IRP  subproblems  are  solved  to  optimality,  and  then  a  subgradient  search  method  is 
employed  to  determine  good  multipliers  for  recombining  the  subproblems.  The  VRP 
is  a  multiperiod  problem  that  can  be  solved  by  decomposing  the  problem  into  single 
period  problems  that  are  solved  and  which  collectively  approximate  the  solution  of  the 
multiperiod  problem.  The  IRP  is  solved  by  examining  the  inter-period  inventory  flow 
and  using  this  to  link  the  periods  for  the  VRP.  Since  total  demand  is  not  necessarily 
met  in  this  formulation,  the  authors  assess  a  penalty  for  any  unmet  demand  for  each 
customer.  The  penalty  value  is  then  used  to  determine  prioritization  for  the  next 
period.  Within  each  period,  each  vehicle  can  only  make  at  most  one  trip,  and  no 
transshipment  is  allowed  between  customers. 
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While  the  VRP  is  a  well  studied  problem,  relatively  little  research  is  dedicated  to 
modeling  and  solving  problems  that  account  for  disruptions  to  the  route.  Mu  et  al. 
[18]  consider  the  case  when  a  vehicle  breaks  down  and  cannot  complete  its  route.  One 
can  model  the  breakdown  as  a  Dynamic/Real-time  Vehicle  Routing  Problem  (DVRP) 
wherein  the  breakdown  is  the  real-time  component,  and  the  evolving  changes  to  the 
schedule  provide  the  dynamic  considerations.  The  focus  of  Mu  et  al.  [18]  is  the 
development  of  a  metaheuristic  that  provides  good  solutions  quickly  due  to  the  time 
sensitivity  of  the  generation  of  a  new  schedule.  Mu  et  al.  [18]  consider  the  case  of 
capacitated  vehicles  without  time  windows  for  deliveries.  In  generating  solutions, 
only  one  extra  vehicle  exists  for  use  since  breakdowns  are  rare  and  not  likely  to 
occur  more  than  once  in  a  day.  When  a  breakdown  occurs,  vehicles  enroute  to  a 
location  complete  the  delivery  before  rerouting  is  considered;  however,  if  a  vehicle 
just  completed  a  delivery  and  has  not  departed,  then  it  is  immediately  available  for 
rerouting.  In  determining  the  priority  of  the  feasible  solutions,  the  preferred  solution 
should  not  include  the  use  of  the  extra  vehicle.  The  next  priority  goes  to  solutions 
that  minimize  total  distance  traveled  by  all  vehicles  for  the  remaining  deliveries.  If 
a  breakdown  occurs  after  the  last  delivery  of  the  vehicle,  then  no  changes  occur  to 
the  vehicle  routes.  The  modeling  of  the  breakdown  occurs  in  a  single  instance  of  the 
VRP.  The  formulation  provided  does  not  attempt  to  solve  the  problem  over  a  time 
horizon.  The  time  of  the  breakdown  is  within  a  one-day  routing  of  deliveries  by  a 
single  supplier  to  multiple  customers. 

Bertazzi  et  al.  [1]  focus  on  addressing  an  IRP  that  has  stochastic  demands  and 
allows  for  stock-out.  In  constructing  the  problem,  each  customer  applies  an  order- 
up-to-level  policy.  This  formulation  does  not  allow  for  backlogging,  and  any  shortage 
between  the  demand  and  order-up-to  level  is  considered  a  loss,  resulting  in  a  penalty. 
The  problem  solved  considers  only  one  vehicle  that  has  a  given  capacity.  The  objec- 
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tive  of  the  formulation  is  to  minimize  the  expected  cost  of  inventory,  penalty,  and 
routing  over  the  time  horizon.  Bertazzi  et  al.  [1]  begin  with  a  dynamic  program¬ 
ming  formulation  and  use  it  to  build  a  hybrid  rollout  algorithm  that  constructs  good 
solutions  to  the  problem  and  addresses  the  curses  of  dimensionality.  The  rollout  algo¬ 
rithm  is  a  useful  heuristic  for  solving  deterministic  and  stochastic  dynamic  programs. 
The  algorithm  developed  is  a  hybrid  because  it  approximates  the  cost-to-go  of  the 
dynamic  program  by  providing  an  exact  solution  to  a  mixed  integer  linear  program 
(MILP)  model.  The  MILP  models  the  deterministic  expectation  of  the  stochastic 
distribution  of  the  demands.  In  solving  the  MILP  model,  Bertazzi  et  al.  [1]  develop 
a  branch-and-cut  algorithm  that  outperforms  existing  expected  value  algorithms. 

Coelho  &  Laporte  [3]  develop  exact  solutions  to  several  classes  of  IRPs.  The 
model  used  is  a  general  formulation  of  the  multi-vehicle  IRP  (MIRP)  that  assumes 
holding  costs  for  both  the  supplier  and  customers  per  time  period.  The  customers 
have  an  inventory  capacity  while  the  supplier  can  meet  the  demand  of  all  customers 
over  the  time  horizon  and  in  each  time  period.  This  prevents  the  inventory  level  at 
the  customers  from  dropping  below  zero  and  prohibits  backlogging.  Each  vehicle  is 
constrained  to  one  route  that  contains  a  subset  of  the  supplier’s  customers.  Since 
several  variations  of  the  MIRP  exist,  Coelho  &  Laporte  [3]  identify  the  following 
variants  as  solvable  by  their  formulation:  1)  quantity  consistency,  2)  vehicle  filling 
rate,  3)  order-up-to  level,  4)  driver  consistency,  5)  driver  partial  consistency,  and  6) 
visit  spacing.  Both  heterogeneous  and  homogeneous  fleets  are  considered.  Coelho 
&  Laporte  [3]  employ  a  general  objective  function  and  constraints  that,  for  each  of 
the  problem  instances  mentioned,  are  ignored  or  take  on  the  form  required  by  the 
particular  class  of  MIRP.  Coelho  &  Laporte  [3]  develop  a  branch-and-cut  algorithm 
that  solves  the  general  formulation  of  the  previous  MIRPs  exactly  by  solving  a  linear 
program  relaxation  via  a  dual  simplex  approach.  When  a  new  best  solution  occurs,  a 
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solution  improvement  algorithm  is  inserted  that  attempts  improvements  by  removing 
or  including  a  customer  in  the  routes. 

The  vehicle  routing  problem  with  stochastic  demand  (VRPSD)  is  an  extension 
of  the  VRP  that  assumes  customer  demand  follows  a  known  distribution.  Novoa  & 
Storer  [19]  study  this  problem  and  model  the  customer  demand  as  following  a  known 
distribution.  The  actual  demand  is  not  known  until  the  vehicle  reaches  the  customer 
and  delivers  supplies.  A  general  solution  methodology  for  the  VRPSD  assumes  an  a 
priori  solution  for  the  vehicle  route.  The  route  is  followed  and  if  the  quantity  in  the 
vehicle  cannot  meet  the  customer  demands  then  extra  trips  to  the  supplier  occur  until 
all  demands  are  satisfied.  The  objective  of  the  formulation  is  to  minimize  total  route 
costs,  including  both  planned  routes  and  extra  routes  resulting  from  additional  returns 
to  the  supplier.  Novoa  &  Storer  [19]  formulate  a  dynamic  program  that  decomposes 
the  problem  into  multiple  stages.  The  program  builds  the  vehicle  route  as  it  visits 
customers  and  determines  whether  or  not  to  return  to  the  supplier  or  proceed  to 
another  customer  based  on  the  current  capacity  of  the  vehicle.  Novoa  &  Storer  [19] 
combine  three  methodologies  in  their  solution  method.  First,  they  introduce  a  rollout 
algorithm.  The  rollout  algorithm  assumes  a  sub-optimal  a  priori  solution,  called  the 
base  sequence,  that  has  a  cost-to-go  that  can  be  easily  approximated.  A  two-stage 
algorithm  is  implemented  that  assumes  there  are  two  more  customers  to  visit  and 
at  each  delivery  the  algorithm  requires  an  update  to  the  base  sequence.  The  second 
method  that  Novoa  &  Storer  [19]  implement  is  a  Monte  Carlo  simulation  (MCS). 
In  computing  the  two-stage  rollout  algorithm  MCS  is  implemented  to  reduce  the 
computational  time  of  the  algorithm  as  it  calculates  the  expected  cost-to-go  for  all 
possible  two-stage  routes  of  the  remaining  unvisited  customers  to  include  trips  back 
to  the  supplier.  The  final  contribution  of  Novoa  &  Storer  [19]  is  the  development 
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of  a  heuristic  solution  method  for  the  single  vehicle  VRPSD.  The  heuristic  uses  a 
stochastic  set-partitioning  model  for  the  initial  formulation  to  build  the  base  sequence. 

1.1.2  Vendor  Managed  Inventory  Replenishment  (VMI). 

Vendor  managed  inventory  (VMI)  replenishment  is  a  business  practice  in  which 
a  central  vendor  monitors  the  inventory  level  of  its  customers  and  determines  the 
quantity  of  commodities  delivered  and  when  deliveries  are  made  to  the  customers 
[13].  This  centrality  of  decision-making  at  the  vendor  level  provides  a  dynamic  that 
influences  the  decision  timeline  and  formulation  of  the  IRP.  The  intent  of  VMI  is  to 
minimize  the  sum  of  inventory  costs  and  transportation  costs  over  the  entire  network. 
In  conventional  inventory  management,  customers  keep  track  of  their  own  inventory 
and  determine  when  to  place  an  order  for  delivery  of  resources.  The  supplier  receives 
the  orders  from  the  customers  and  then  prepares  the  orders  for  delivery  using  its 
vehicles. 

The  disadvantages  of  conventional  inventory  management  include  non-uniformity 
of  order  demand  and  customer  misinterpretation  of  urgent  order  criteria  [13].  The 
first  disadvantage  addresses  the  issue  of  orders  arriving  simultaneously  from  cus¬ 
tomers  due  to  customers  checking  their  inventories  at  the  same  time  and  then  placing 
orders.  The  second  issue  accounts  for  customers  biasing  the  priority  of  their  order 
by  inflating  its  importance.  This  prevents  true  identification  of  urgent  orders.  The 
advantages  of  VMI  include  the  vendor  controlling  inventory  at  the  customers  and  the 
delivery  schedule  [14],  By  controlling  the  inventory,  the  vendor  determines  when  to 
make  deliveries  and  is  able  to  direct  a  more  uniform  schedule  of  deliveries  and  better 
manage  its  vehicle  fleet.  Furthermore,  the  vendor  determines  the  criteria  for  delivery 
and  can  identify  emergency  situations  when  inventory  levels  are  critical.  Customers 
benefit  from  VMI  as  well  because  the  central  control  of  inventory  increases  reliability 
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of  not  having  shortages  and  they  no  longer  have  to  devote  resources  to  inventory 
management  [14]. 

The  implementation  of  VMI  enables  the  vendor  to  determine  the  demand  history 
of  the  customers  [13].  The  vendor  combines  this  with  the  knowledge  of  customer 
locations  to  determine  low-cost  routing  and  vehicle  management  in  order  to  meet 
demand.  Kleywegt  et  al.  [13]  suggest  that  such  routing  and  vehicle  assignment  can 
include  capacitated  and  uncapacitated  vehicle  loading.  Successful  implementation  of 
VMI  requires  timely  and  accurate  information  regarding  customer  inventory  [14]. 

Kleywegt  et  al.  [13]  consider  an  IRP  with  the  special  case  of  direct  deliveries  and  a 
single  commodity.  They  consider  the  customers  as  having  a  capacity,  and  the  vendor 
as  having  a  fleet  of  homogeneous  vehicles  with  a  known  capacity.  Each  customer  has 
a  demand  with  a  probability  distribution  known  to  the  vendor.  Kleywegt  et  al.  [13] 
formulate  the  problem  such  that  the  vendor  can  measure  the  inventory  level  at  all 
customers  at  any  time  and  determine  when  and  how  much  commodity  to  deliver.  In 
the  general  case,  delivery  to  multiple  customers  can  be  combined  on  a  route;  however, 
this  is  reduced  to  a  direct  delivery  to  a  single  customer  on  each  route.  The  routes 
that  Kleywegt  et  al.  [13]  construct  can  all  be  traversed  in  less  than  a  day,  which 
makes  possible  multiple  deliveries  to  a  customer  in  a  day.  This  also  means  that  the 
entire  vehicle  fleet  is  available  at  the  beginning  of  every  day.  Kleywegt  et  al.  [13] 
formulate  the  IRP  as  a  Markov  decision  process  (MDP)  over  an  infinite  time  horizon 
and  apply  dynamic  programming  to  solve  the  problem.  They  define  a  state  as  the 
current  inventory  at  each  customer;  the  action  space  to  contain  the  set  of  decisions 
to  deliver  an  amount  of  commodity  to  each  customer;  and  the  reward  function  to 
account  for  the  revenue  gained  for  positive  difference  between  commodity  required, 
commodity  delivered,  and  commodity  in  inventory,  minus  the  holding  cost  of  left 
over  inventory  at  the  customer.  A  negative  difference  in  inventory  is  discouraged  by 


applying  a  penalty  function  to  any  shortages  at  the  customers.  The  known  probability 
function  of  demand  enables  a  Markov  transition  function  that  governs  the  transition 
function  from  state  to  state  [13].  The  stochastic  nature  of  the  demand  means  that 
the  functional  equation  is  the  maximization  of  the  expected  total  discounted  gain, 
i.e. ,  revenue  minus  costs,  for  all  customers  over  an  infinite  horizon. 

The  work  conducted  by  Kleywegt  et  al.  [14]  extends  the  work  of  Kleywegt  et  al. 
[13]  by  relaxing  the  direct  delivery  constraint  in  the  VRP  subproblem.  In  this  formu¬ 
lation  of  the  VRP  the  available  routes  expand  to  also  consider  multiple  customers  per 
route.  Vehicles  only  complete  one  route  per  day,  all  routes  are  completed  in  a  day,  and 
all  vehicles  are  available  for  use  each  day.  Customer  daily  demands  are  independent 
random  variables  that  follow  a  known  probability  distribution.  Each  customer  has  a 
finite  inventory  capacity  that  is  not  constant  between  customers.  Stochastic  customer 
demands  allow  for  shortages,  and  a  penalty  function  is  imposed.  However,  shortages 
are  not  backlogged  and  represent  a  total  loss  to  the  vendor.  Because  of  their  specific 
motivating  problem,  Kleywegt  et  al.  [14]  do  not  consider  an  inventory  holding  cost 
to  the  vendor,  only  to  the  customer.  They  also  formulate  this  problem  as  having 
an  infinite  horizon  given  the  nature  of  the  industry  they  are  modeling.  The  IRP  is 
formulated  as  a  discrete  time  MDP.  The  state  of  the  system  is  the  inventory  at  each 
customer  at  time  t;  the  decision  space  is  the  set  of  inventory  that  is  delivered  to  each 
customer  at  time  t  given  that  customers  can  be  combined  on  a  route  given  vehicle 
capacity;  and  the  decision  taken  identifies  the  amount  of  inventory  delivered  to  each 
customer.  Kleywegt  et  al.  [14]  assume  that  no  inventory  is  used  by  the  customer  be¬ 
tween  the  time  it  is  measured  and  when  inventory  is  delivered  by  the  vendor.  A  known 
Markov  transition  function  results  from  the  fact  that  the  only  stochastic  element  is 
the  customer  demand,  which  has  a  known  probability  distribution.  The  revenue  for  a 
positive  net  difference  between  commodity  required,  commodity  delivered,  and  com- 
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modity  in  inventory  less  the  holding  cost  of  inventory  still  at  the  customer  and  any 
shortage  penalty  cost  comprise  the  reward  function.  The  functional  equation  for  this 
problem  is  the  maximization  of  the  expected  discounted  gain  for  all  customers  over 
an  infinite  horizon. 

1.2  The  Military  Inventory  Routing  Problem 

The  motivating  problem  of  interest  in  this  paper  is  the  sustainment  of  an  in¬ 
fantry  brigade  combat  team  (BCT)  in  a  combat  environment  through  the  use  of 
cargo  unmanned  aerial  systems  (CUAS).  An  austere  combat  environment,  like  that 
of  Afghanistan,  is  of  particular  interest.  Our  purpose  is  to  develop  a  dynamic  pro¬ 
gramming  formulation  that  exactly  solves  a  small  instance  of  the  MILIRP.  Our  for¬ 
mulation  prescribes  an  optimal  delivery  policy  with  which  the  United  States  Army 
(Army)  is  able  to  maximize  the  tonnage  of  supplies  delivered  by  a  fleet  of  homoge¬ 
neous  CUAS  in  order  to  meet  the  supply  demands  of  an  IBCT  with  combat  outposts 
(COPs)  dispersed  throughout  its  area  of  operation  (AO).  Key  to  our  formulation  is 
the  modeling  of  the  possible  destruction  of  the  CUAS.  Our  optimal  delivery  policy 
must  consider  the  number  of  available  CUAS,  the  impact  of  losing  the  CUAS  on  fu¬ 
ture  deliveries,  and  an  environment  with  changing  threat  conditions  that  the  IBCT 
encounters.  This  allows  us  to  investigate  changes  in  the  decisions  to  deliver  to  COPs 
as  the  probability  of  successfully  delivering  to  the  COPs  decreases.  The  ultimate  end 
state  of  our  research  is  to  inform  the  Army  as  it  determines  the  number  of  CUAS  to 
have  in  a  fleet  of  vehicles  with  the  purpose  of  complete  sustainment  of  an  IBCT  over 
a  12-month  deployment  while  facing  varying  threat  conditions. 

The  Army  identifies  sustainment  as  a  key  function  that  commanders  must  use 
in  order  to  accomplish  missions  [9].  Moreover,  the  Army  defines  sustainment  as 
the  actions  necessary  to  provide  the  freedom  to  act,  extend  the  reach  of  missions, 
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and  enable  prolonged  operations.  Sustainment  is  essential  to  military  operations  [9]. 
The  Army  identifies  six  principles  of  sustainment.  The  following  four  principles  have 
particular  relevance  to  this  paper:  1)  responsiveness,  2)  simplicity,  3)  economy,  and  4) 
survivability.  Responsiveness  enables  commanders  the  ability  to  set  and  maintain  a 
tempo  without  fatigue,  rotation  of  forces,  and  extending  operational  reach.  Simplicity 
provides  clear,  concise,  repeatable  action  that  standardizes  procedures  and  establishes 
a  repeatable  rhythm.  Efficient  economic  allocation  and  prioritization  of  resources 
combine  to  achieve  the  greatest  affect  and  create  economy.  This  includes  the  use 
of  contract  services  to  make  the  most  of  the  use  of  military  resources.  Survivability 
is  the  protection  of  personnel,  materiel,  and  resources  against  hostile  action  while 
ensuring  mission  accomplishment  [10]. 

In  the  Army  hierarchy,  the  brigade  combat  team  (BCT)  is  the  organizational  level 
in  which  self-contained  operations  are  conducted.  A  BCT  owns  all  required  assets 
necessary  for  its  operational  mission  set.  The  brigade  ensures  that  all  subordinate 
units  have  the  requirements  necessary  for  successful  operations.  The  brigade  support 
battalion  (BSB)  is  the  element  responsible  for  ensuring  that  the  subordinate  units 
within  the  brigade  have  all  necessary  supplies  [7]. 

The  responsibility  for  planning  sustainment  operations  for  the  BCT,  which  in¬ 
cludes  supply,  belongs  to  the  BCT  supply  officer  and  includes  the  operations  officer, 
supply  officer,  and  support  operations  officer  of  the  BSB.  The  BSB  has  the  responsi¬ 
bility  for  planning,  preparing,  and  overseeing  sustainment  operations  in  the  brigade 
AO  [8].  This  requires  coordination  with  and  monitoring  subordinate  units’  supply 
requirements.  Distribution  management  within  the  BSB  synchronizes  and  prioritizes 
the  execution  of  supply  operations  within  the  BCT.  This  is  accomplished  through  the 
use  of  automated  systems  and  regular  reporting  by  subordinate  units  which  enables 
the  planning  of  sustainment  operations  for  the  BCT  [8].  The  central  monitoring  and 
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distribution  of  supply  inventory  that  Army  logisticians  utilize  mirrors  the  civilian 
business  practice  of  VMI.  Like  the  civilian  sector,  this  reduces  wasted  resources  be¬ 
cause  the  BSB  has  the  ability  to  move  supplies  where  they  are  needed  using  a  limited 
capacity  fleet  of  vehicles. 
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II.  Background 


2.1  Army  Resupply 

The  Army  conducts  military  operations  in  a  variety  of  operational  environments. 
These  environments  often  contain  harsh,  rugged  terrain  that  includes  mountains, 
deserts,  and  jungles  [11].  The  Army  expects  to  continue  to  operate  in  remote  loca¬ 
tions  characterized  by  these  conditions.  As  a  result,  Army  logistics  and  sustainment 
operations  must  provide  support  across  a  dispersed  area  of  operation  (AO).  In  an  in¬ 
dependent  study  conducted  for  the  Army,  General  Dynamics  Information  Technology 
[12]  identifies  the  following  five  challenges  to  resupply  missions:  1)  demand  for  large 
quantities  of  supplies  across  area  of  operations,  2)  effects  of  enemy  threat  and  action, 
3)  weather,  terrain,  and  poor  infrastructure,  4)  availability  of  distribution  assets,  and 
5)  flexibility  to  respond  to  changes  in  the  operational  environment. 

Military  resupply  missions  rely  heavily  on  the  use  of  ground  lines  of  communica¬ 
tions  (GLOCs).  However,  the  existance  and  quality  of  infrastructure  as  well  as  the 
potential  harshness  of  the  environment  affect  the  utility  of  GLOCs  for  resupply.  The 
lack  of  improved  roadways  prevents  vehicular  travel,  and  pack  animals  are  often  used 
to  traverse  the  rugged  terrain.  Moreover,  GLOCs  are  extremely  vulnerable  to  attacks 
by  improvised  exploding  devices  (IEDs)  and  other  anti-access  technologies  which  re¬ 
strict  freedom  of  movement  [12],  For  example,  the  targeting  of  convoys  along  GLOCs 
with  IEDs  accounted  for  65%  of  all  U.S.  deployed  fatalities  between  November  2002 
and  March  2009,  with  18%  occurring  during  sustainment  operations  [12],  Host-nation 
assets  have  been  used  to  provide  resupply;  however,  this  incurs  added  security  risks 
and  vulnerabilities  that  can  delay  the  availability  of  the  supplies  or  compromise  their 
delivery  [12], 
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The  remoteness  of  most  of  the  COPs  in  Afghanistan  makes  GLOCs  difficult  if 
not  impossible  to  use.  Air  assets,  i.e.,  helicopters,  provide  an  alternative  means  for 
resupply.  Due  to  the  nature  of  austere  operational  environments,  logisticians  prefer  to 
use  such  aerial  assets.  An  Army  simulation  determined  that  approximately  a  quarter 
of  division’s  supply  requirements  can  be  met  by  current  air  assets  [12]. 

Limitations  do  exist  for  air  resupply  resulting  from  the  terrain  and  to  a  greater 
extent,  the  weather.  During  harsh  and  extreme  weather,  pilots  cannot  fly  due  to 
limited  visibility,  especially  at  higher  altitudes  where  lower  air  density  negatively  im¬ 
pacts  helicopter  responsiveness.  Moreover,  as  the  Army  tries  to  provide  more  supplies 
through  aerial  means,  the  threat  of  man-portable  air  defense  systems  (MANPADS) 
becomes  a  greater  concern.  Helicopters  are  most  vulnerable  during  landing  and  take¬ 
off  at  the  COPs,  often  fly  in  pairs  to  mitigate  the  threat  of  MANPADS,  and  may  even 
have  armed  escort  aircraft.  At  times,  even  implementing  risk  mitigation  techniques 
are  not  enough,  and  risk  reduction  occurs  by  denying  resupply  missions  until  the 
situation  in  the  area  improves. 

Due  to  the  high  operational  tempo  (OPTEMPO),  combat  mission  support  for 
helicopters  takes  precedence  over  sustainment  missions.  This  limits  the  number  and 
availability  of  helicopters  for  resupply  operations,  causing  both  unpredictability  in 
support  for  resupply  and  the  possibility  of  last  minute  changes  to  asset  allocation 
[12].  This  lack  of  consistent  availability  of  military  helicopters  led  to  the  use  of  “Jingle 
Air” ,  a  group  of  contractors  that  operate  helicopters  in  support  of  resupply  missions. 
During  a  deployment  to  Afghanistan  in  2008-2009,  the  801si  BSB  moved  more  supplies 
through  these  contracts  than  by  Army  and  Air  Force  means  combined  [12].  However, 
these  missions  could  only  be  conducted  over  eight  hour  shifts,  during  daytime,  and 
under  conditions  within  the  limits  of  the  instrumentation.  Another  limitation  of 
using  contractors  to  conduct  resupply  missions  is  the  set  of  allowable  supplies  that 
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the  contractors  may  move.  Therefore,  the  use  of  convoys  along  vulnerable  GLOCs 
continues  for  intra-theater  sustainment  operations  due  to  military  helicopters  meeting 
operational  needs  and  the  limitations  of  civilian  contractors  [12]. 

2.2  Role  of  the  UAS 

Current  sustainment  operations  require  aviation  assets  and  ground  convoys  to 
successfully  meet  the  supply  needs  of  the  COPs.  As  a  way  to  mitigate  the  use  of 
convoys  and  rotary  aircraft,  the  Army  is  investigating  the  employment  of  rotary  cargo 
unmanned  aircraft  systems  (CUAS)  to  conduct  the  supply  missions.  The  use  of  rotary 
CUAS  has  particular  interest  given  most  of  the  COPs  do  not  have  the  physical  space 
for  a  runway  nor  the  ability  to  secure  one.  By  introducing  rotary  CUAS,  the  required 
number  of  ground  convoys  decreases,  allowing  for  a  better  utilization  of  combat  and 
support  units.  Moreover,  rotary  CUAS  do  not  have  the  same  flight  ceiling  restrictions 
since  cabin  pressure  does  not  matter.  This  enables  the  attainment  of  higher  altitudes, 
reducing  the  MANPAD  threat  and  possibly  enabling  shorter  supply  routes  and  quicker 
delivery  of  supplies  to  the  COPs.  General  Dynamics  Information  Technology  [12] 
reports  that  manned  rotary  craft  suffer  from  adverse  weather  limitations  that  range 
from  poor  visibility  constraints  to  dangerous  flight  conditions  for  pilots.  In  contrast, 
CUAS  is  ideal  for  these  types  of  conditions  because  soldiers  will  not  be  affected  by 
the  flying  conditions.  This  lack  of  limitation  for  CUAS  widens  the  delivery  window  of 
supplies  and  enables  commanders  and  logisticians  to  plan  more  sustainment  missions 
than  possible  with  manned  rotary  aircraft. 

Sustainment  missions  only  account  for  a  portion  of  the  mission  requirements  of 
Army  Aviation.  Additionally,  the  Army  does  not  have  enough  vertical  lift  capability 
to  meet  demands.  General  Dynamics  Information  Technology  [12]  notes  that  42%  of 
the  Army’s  Chinooks,  its  heavy  lift  helicopters,  were  deployed  during  2008.  With  less 


15 


than  half  of  its  total  force  available  to  combat  commanders,  mission  prioritization  be¬ 
comes  paramount.  A  survey  of  deployed  soldiers  reveals  that  only  about  5.7%  believed 
that  helicopter  requests  for  sustainment  missions  would  be  supported,  with  mission 
priority  as  the  main  reason  given  for  the  request  denial  [12].  The  implementation 
of  rotary  CUAS  dedicated  to  sustainment  missions  provides  two  benefits.  First,  it 
provides  a  dedicated  sustainment  platform  and  fleet  of  assets  to  meet  the  sustainment 
needs  of  the  combat  commander.  This  enables  logisticians  to  have  a  known  and  fixed 
number  of  aircraft,  i.e,  CUAS,  available  for  planning  and  utilization  to  meet  sustain¬ 
ment  needs.  Second,  CUAS  logistic  support  frees  up  helicopters  for  other  types  of 
missions,  in  particular  combat,  personnel  transport,  and  medical  operations. 

High  operational  tempo  (OPTEMPO)  and  reliance  on  contractors  causes  vari¬ 
ability,  uncertainty,  and  a  lack  of  responsiveness  in  sustainment  mission  support  and 
execution.  On  the  one  side,  operational  requirements  cause  sustainment  missions  to 
be  canceled  and  support  to  be  denied.  Contractor  support  requires  long-term  regi¬ 
mented  planning  and  coordination.  This  limits  flexibility  and  prevents  the  support 
of  immediate  and  emergency  supply  requests.  General  Dynamics  Information  Tech¬ 
nology  [12]  provides  the  following  quote  by  an  Infantry  officer,  “If  the  support  is  not 
anticipated  (more  than  72  hours  out),  then  you  are  not  getting  the  support.  There  is 
no  immediate  resupply.”  A  dedicated  CUAS  capability  controlled  by  the  logisticians 
can  help  mitigate  these  issues.  As  previously  stated,  a  dedicated  supply  capability 
eliminates  the  competing  requirements  for  the  aircraft  and  allows  the  heavy  lift  heli¬ 
copters  to  focus  on  operational  missions  and  the  CUAS  to  focus  on  sustainment.  A 
dedicated  CUAS  controlled  by  logisticians  also  mitigates  the  issue  of  responsiveness. 
While  planning  sustainment  is  required,  due  to  the  high  OPTEMPO  that  combat 
units  operate  within,  supply  requirements  will  change  unexpectedly.  Military  logisti¬ 
cians  understand  this  and  have  the  ability  to  change  the  sustainment  mission  plans 
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accordingly  to  meet  the  immediate  demands  of  the  units  without  having  to  go  through 
the  protocols  required  by  using  contractor  services  [12]. 

Among  the  supplies  moved  across  the  battlefield,  bulk  liquids  is  the  largest  type 
that  must  be  transported.  As  COPs  grow  in  size  and  number,  more  fuel  has  to  be 
transported.  In  November  of  2007  it  was  noted  that  it  required  an  entire  division’s 
heavy  lift  capability  to  supply  one  brigade’s  bulk  fluid  requirement  [12].  In  order 
to  transport  bulk  fluid  by  air,  all  other  missions  would  be  suspended.  This  is  not  a 
realistic  option  and,  as  such,  ground  convoys  on  GLOCs  are  necessary  to  transport 
bulk  liquids.  As  stated  earlier,  GLOCs  are  vulnerable  to  IEDs  and  small  arms  fire. 
Since  most  bulk  liquids  are  fuel,  this  becomes  highly  problematic  and  dangerous  for 
the  ground  convoys  and  helicopters.  A  CUAS  capability  alleviates  the  reliance  on 
GLOCs  and  enables  the  heavy  lift  helicopters  to  conduct  operational  missions.  Also, 
CUAS  helps  to  distance  personnel  from  the  volatile  liquids  during  transportation  and 
makes  transport  safer  [12]. 
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III.  Problem  Definition 


3.1  Problem  Description 

We  consider  the  case  where  an  Infantry  Brigade  Combat  Team  (BCT)  is  operating 
in  an  austere  combat  environment.  The  BCT  is  responsible  for  security  in  its  AO  and 
disperses  its  subordinate  units  in  COPs  throughout  the  AO.  Figure  1  depicts  an  AO 
in  Southeastern  Afghanistan  and  shows  how  an  BCT  may  locate  the  COPs  within 
it.  The  BCT  headquarters  and  BSB,  represented  by  the  black  diamond,  are  both 
located  in  the  center  of  the  AO  to  facilitate  distribution  of  supplies.  The  subordinate 
units  are  located  at  the  COPs,  small  colored  diamonds,  throughout  the  AO  in  a 
manner  consistent  with  the  BCT’s  overall  counterinsurgency  mission.  Supplies  are 
distributed  to  the  battalions  by  the  BSB.  Since  battalions  do  not  possess  air  assets, 
distribution  to  the  COPs  occurs  centrally  through  the  BSB  using  a  fleet  of  CUAS 
located  with  the  BSB.  Supplies  are  distributed  to  F  COPs  by  means  of  V  identical 
CUAS  systems.  A  location  specific  commodity  storage  capacity  C%  exists  at  each  of 
the  COPs.  Each  CUAS  is  identical  and  has  a  capacity  G.  The  demand  ult  for  each 
COP  is  stochastic  in  nature  and  varies  from  day-to-day,  depending  on  the  combat 
operations  conducted.  The  amount  of  supplies  delivered  each  trip  taken  by  a  CUAS 
is  di,  which  specifies  the  supplies  delivered  to  COP  i  during  the  trip.  When  utilizing 
rotary  air  assets,  it  is  infeasible  to  combine  multiple  deliveries  where  more  than  one 
location  must  be  visited.  This  imposes  a  direct  delivery  formulation  of  the  routing 
constraint.  Furthermore,  it  can  easily  be  shown  that  any  delivery  that  has  less  than  a 
full  load  is  dominated  by  delivery  at  full  vehicle  capacity.  The  direct  delivery  routing 
formulation  results  in  the  di  for  each  CUAS  equaling  G.  Time  is  modeled  discretely 
with  time  periods  t  —  0, 1, ...,  T.  Inventory  level  Xu  represents  the  inventory  at  COP 
%  at  time  t.  All  round-trip  routes  used  in  the  model  can  be  traversed  in  less  than  one 
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Figure  1.  BSB  and  COP  Locations  throughout  the  IBCT  AO 
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time  period.  The  time  required  to  execute  a  round-trip  for  a  given  route  includes 
the  time  it  takes  to  load  the  CUAS,  travel  to  the  COP,  unload,  return,  and  refuel. 
All  of  these  activities  are  accomplished  within  one  time  period,  which  we  assume  is 
less  than  twelve  hours.  This  enables  multiple  routes  per  day  for  each  CUAS.  It  also 
means  that  all  CUAS  are  available  at  the  beginning  of  each  day.  For  the  range  of  the 
CUAS  to  not  be  an  issue,  the  furthest  direct  delivery  from  the  BSB  to  a  COP  must 
be  within  the  operational  radius  of  a  fully  loaded  CUAS.  Refueling  occurs  at  the  BSB 
depot  at  the  end  of  a  route,  thereby  mitigating  the  need  to  calculate  maximum  route 
lengths  or  legs  before  refueling.  Time  windows  for  deliveries  are  not  considered  in 
this  problem  since  units  are  always  available  at  the  COPs  to  receive  the  supplies,  and 
there  is  no  other  delivery  location  for  the  supplies,  as  the  COPs  represent  a  terminal 
node. 

Most  VRP  and  IRP  formulations  consider  different  costs  associated  with  traversing 
a  route  or  maintaining  inventory  levels.  In  our  formulation,  these  costs  are  not  of 
interest.  While  fuel,  operation,  and  maintenance  costs  exist,  these  costs  are  not  used 
to  inform  commanders  in  the  decision  of  how  to  ensure  supplies  are  delivered  to  their 
subordinates.  Commanders  are  more  concerned  with  the  successful  delivery  to  the 
subordinate  units.  For  this  reason,  the  cost  of  traveling  a  particular  route  is  set 
aside  in  this  research.  Moreover,  there  is  no  cost  for  holding  inventory.  This  occurs 
because  there  is  no  disincentive  for  units  to  have  supplies  in  inventory.  Shortages 
are  considered  in  this  model;  however,  a  penalty  function  is  not  explicitly  used  to 
punish  shortages.  The  reward  function  is  the  utility  gained  by  delivering  thousands 
of  pounds  of  inventory  by  CUAS  to  each  COP  during  time  t.  When  a  shortage 
occurs,  we  assume  a  ground  convoy  delivers  enough  inventory  to  bring  the  COP  to 
capacity.  This  results  in  a  lost  opportunity  of  potential  utility  gained  by  having  a 
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CUAS  conduct  the  delivery  and  serves  as  the  de  facto  penalty  since  the  objective  in 
this  formulation  is  the  maximization  of  utility  gained  by  CUAS  delivery. 

3.2  Tessellation  of  the  Geographic  Region 

In  our  formulation  of  the  MILIRP  we  have  an  BCT  with  subordinate  units  dis¬ 
persed  through  an  area  of  operations.  The  brigade  headquarters  and  its  BSB  are 
located  near  the  region’s  geographic  center  to  facilitate  responsive  support  to  all 
units.  This  location  serves  as  the  supplier  or  depot  in  our  problem  formulation,  from 
where  CUAS  are  dispatched  to  deliver  supplies  to  subordinate  units  (i.e.,  customers). 
These  subordinate  units  are  located  at  COPs  spread  through  the  region’s  geography 
based  on  the  tactical  need  for  units  to  conduct  operations.  We  restrict  our  attention 
to  COPs  within  the  maximum  fully  loaded  radius  of  the  CUAS,  as  measured  via 
Euclidean  distance. 

In  order  to  develop  tractable  instances  to  test  our  formulation  of  the  MILIRP,  we 
elect  to  tesscllate  a  representative  brigade-sized  area  of  operations  into  finite  elements 
using  a  mesh  of  uniformly-sized  regular  polygons.  This  discretization  yields  two 
benefits.  First,  it  reduces  the  number  of  paths  within  the  area  of  operations  between 
the  CUAS’s  base  to  a  specific  demand  from  an  infinite  set  to  a  combinatorial  set  that 
depends  on  the  granularity  of  the  discretization.  Second,  it  allows  for  the  aggregation 
of  risk  to  the  CUAS  in  its  flight  over  a  discrete  region  based  on  enemy  threat  and 
challenging  weather  conditions  (e.g.,  updrafts  due  to  terrain). 

For  this  study,  we  choose  to  discretize  the  brigade-sized  area  of  operations  using  a 
mesh  of  uniformly-sized  regular  hexagons.  Although  square,  rhombus,  and  triangular 
meshes  are  also  feasible,  Yousefi  &  Donohue  [22]  demonstrated  that,  for  the  same 
granularity  of  discretization,  a  hexagonal  mesh  exhibits  the  relative  advantage  of 
enabling  clustering  in  more  directions.  In  other  words,  a  CUAS  at  one  hexagonal  cell 
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has  six  directions  from  which  to  select  its  next  location,  whereas  the  cells  resulting 
from  a  square  (or  rhombus)  and  triangular  tessellation  have  only  four  and  three, 
respectively.  Thus,  the  hexagonal  mesh  discretizes  the  space  into  uniformly-sized  and 
uniformly-shaped  cells  allowing  for  CUAS  flight  paths  that  more  accurately  represent 
the  variability  inherent  in  a  continuous  region.  As  in  Lunday  et  al.  [16],  we  use  a 
horizontal  orientation  for  our  hexagonal  mesh,  wherein  two  opposing  sides  of  the 
hexagonal  cells  lie  on  an  East- West  orientation.  For  our  instances,  we  utilize  regular 
hexagons  with  a  side  length  of  g  equal  to  3.69  km,  resulting  in  cell  areas  of  9.59  km2 
each.  Figure  2  illustrates  the  result  of  this  tessellation,  with  the  cell  containing  the 


Figure  2.  Tessellation  of  Geographic  Region 


BSB  designated  using  a  black  diamond,  and  cells  that  contain  COPs  designated  using 
shading  and  a  colored  diamond  within  to  indicate  the  relative  location  of  the  COP. 
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3.3  Threat  Map 


When  considering  routing  problems  the  military  analyzes  both  the  available  paths 
and  the  threat  of  enemy  action.  The  military  identifies  areas  as  having  a  certain  threat 
level  based  on  the  probability  of  enemy  action.  For  purposes  of  threat  labeling,  an 
area  is  represented  by  a  hexagonal  cell.  The  cumulative  effect  of  the  probabilities 
of  enemy  action  at  each  leg  within  a  route  is  used  as  a  surrogate  for  determining 
the  probability  of  successfully  transiting  between  points  on  a  route  (i.e,  between  two 
adjacent  hexagons).  In  this  formulation,  we  consider  two  threat  levels:  high  and  low. 
We  denote  ay  as  the  probability  of  successfully  transitioning  between  two  low  threat 
areas.  We  denote  um  as  the  probability  of  successfully  transitioning  between  a  low 
threat  and  high  threat  area  in  either  direction.  We  denote  c Oh  as  the  probability  of 
successfully  transitioning  between  two  high  threat  areas.  Of  note,  the  relationship 
between  ay,  u}m,  and  c Oh  is  0Jh  <  com  <  ay. 

The  threat  of  enemy  action  is  not  constant  over  time.  In  a  report  by  Cordesman 
et  al.  [4],  the  data  of  incidents  of  enemy  attacks  over  the  past  three-and-a-half  years 
in  Afghanistan  exhibit  cyclic  behavior.  For  our  instances,  we  use  this  data  to  create 
a  series  of  threat  maps  that  model  the  probability  of  enemy  action  across  the  brigade 
area  of  operations.  To  increase  realism,  we  group  the  threat  maps  by  season  and  use 
them  to  characterize  the  change  in  the  threat  due  to  enemy  action  over  the  course 
of  a  year,  with  a  total  of  M  threat  maps  that  approximate  the  threat  conditions  an 
BCT  may  encounter  over  a  12-month  deployment. 

Threat  maps  that  represent  the  winter  months  have  few  high  threat  areas.  The 
number  of  high  threat  areas  increase  until  the  summer  months,  which  have  the  most 
high  threat  areas.  The  fall  months  illustrate  a  drop  in  high  threat  areas  with  a 
more  marked  drop  going  into  winter.  The  transition  between  threat  maps  follows  a 
Markovian  transition  function.  Regardless  of  the  current  threat  map,  the  transition 
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to  the  next  map  is  governed  by  a  probability  transition  matrix  Q  that  dictates  the 
one-step  transition  probability  to  all  other  threat  maps.  This  transition  matrix  itself 
changes  given  the  current  season;  meaning  transitions  of  threat  maps  characteristic 
of  winter  months  to  threat  maps  characteristic  of  summer  months  is  less  likely  than 
transitions  to  other  characteristically  winter  maps  or  spring  maps.  In  determining 
which  areas  are  high  and  low  threat,  we  employ  military  terrain  analysis  techniques 
[6] .  These  techniques  use  an  understanding  of  how  terrain  influences  the  movement  of 
enemy  forces  and  the  locations  where  they  concentrate  their  actions  against  friendly 
forces.  BCTs  conduct  this  type  of  analysis  on  a  daily  basis  in  order  to  determine 
operational  missions  as  well  as  the  routing  of  sustainment  operations.  Moreover,  this 
analysis  influences  the  actual  probabilities  implemented  within  the  one-step  transition 
matrix. 

3.4  Routing  Formulation 

The  vehicle  routing  problem  (VRP)  is  a  subproblem  to  the  MILIRP.  In  our  prob¬ 
lem,  the  vehicles  are  CUAS  and  travel  is  conducted  through  the  tessellated  airspace. 
To  limit  the  available  routes,  the  airspace  considered  lies  within  the  boundary  of  the 
BCT  area  of  operations.  A  brigade  controls  all  activity  within  its  AO  and  deconflicts 
all  movement.  This  mitigates  the  coordination  and  deconfliction  of  airspace  as  the 
BSB  conducts  the  sustainment  operations  on  behalf  of  the  brigade.  Internal  decon¬ 
fliction  of  airspace  in  this  formulation  assumes  that  routing  based  on  threat  of  enemy 
action  is  the  only  factor  and  that  friendly  actions  do  not  affect  available  routing  op¬ 
tions.  Furthermore,  routes  are  created  by  traversing  adjacent  hexagonal  cells  that 
form  the  tessellation  mesh  used  to  discretize  the  BCT  AO.  We  solve  instances  of  the 
VRP  a  priori  and  then  populate  the  IRP  and  solve  the  dynamic  program  for  the 
optimal  policy. 
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We  implement  Dijkstra’s  algorithm  on  a  network  representation  of  the  hexagonal 
cells  that  describe  the  BCT  AO.  Each  hexagon  represents  an  area  to  which  a  rotary 
CUAS  can  travel.  In  this  formulation  each  hexagon  is  a  node  and  is  adjacent  to  at 
least  one  other  node  and  no  more  than  six  nodes.  The  node  network  that  represents 
the  BCT  AO  is  a  fully  connected,  undirected  graph  and  contains  q  nodes  and  g  arcs. 
The  threat  level  of  each  node  pair  in  the  network  determines  the  arc  distance  between 
them;  which  represents  the  probability  of  successfully  traversing  the  arc  between  the 
nodes  in  the  network.  For  our  problem,  we  assume  the  movement  between  nodes 
to  depend  only  on  the  nodes  being  traversed  and  to  be  independent  of  any  previous 
movement  between  nodes.  This  Markovian  property  is  leveraged  when  determining 
the  shortest  path  between  any  two  nodes.  The  sequential  movement  between  nodes 
can  be  expressed  as  the  product  of  arc  distances,  p,  in  the  route. 

The  multiplicative  property  of  independent  probabilities  enables  us  to  compute 
the  probability  of  successfully  traversing  a  route  r  from  the  BSB  to  a  COP  as: 


II  p^ky  (3-1) 

C j,k)er 


Note  that  hireling  the  path  r  that  maximizes  Equation  3.1  is  equivalent  to  Ending  the 
path  r*  such  that 


r  G  argnrax 


IT 

(yfc)er 


which  is  also  equivalent  to  finding  the  path: 


(3.2) 


r*  G  argmin 


IT  P(j>k) 

(j,k)er 


(3-3) 
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Applying  a  logarithmic  transformation  we  obtain 


r*  E  arg  min  <  —  Z)  Zn(P0\*))  (  ■  (3-4) 

^  (j-k)er  J 

This  allows  us  to  denote  the  arc  lengths  as  — Zn(p(j  m)  and  utilize  Dijkstra’s  Algorithm 
to  identify  the  shortest  path  through  such  a  network,  which  is  equivalent  to  finding 
the  path  with  the  maximum  probability  of  successful  delivery  of  the  commodity.  We 
denote  the  probability  of  success  for  an  outgoing  route  from  the  BSB  to  COP  i  using 
threat  map  m  as  This  is  used  in  the  dynamic  programming  formulation  of  the 
main  IRP. 


Figure  3.  Route  with  Maximum  Probability  of  Success  for  Delivery  to  COP  1  on 
Winter  Threat  Map  1 

Dijkstra’s  algorithm  provides  an  optimal  route  to  each  COP  from  the  BSB  for 
every  threat  map.  As  the  areas  of  high  threat  change  and  the  number  of  high  threat 
areas  change  between  each  map,  different  routes  are  constructed.  Figures  3  and  4 
provide  examples  of  implementing  Dijkstra’s  algorithm  in  our  problem.  The  low 
threat  areas  have  hexagons  that  are  colored  green  and  the  high  threat  areas  are 
colored  red.  The  BSB  is  a  green  and  black  checkered  hexagon  and  has  a  low  threat 
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Figure  4.  Route  with  Maximum  Probability  of  Success  for  Delivery  to  COP  1  on 
Winter  Threat  Map  2 


of  enemy  action.  The  COP  is  colored  yellow  and  green  and  also  has  a  low  threat  of 
enemy  action.  Figure  3  shows  the  route  with  the  maximum  probability  of  success  from 
the  BSB,  in  the  top  left,  to  COP  1,  in  the  bottom  right.  The  probability  of  success 
for  this  route  is  0iii=(c <y)14  since  14  low-low  arcs  are  traversed.  Figure  4  shows  the 
optimal  probability  of  success  route  from  the  BSB  to  the  same  COP  using  a  different 
threat  map.  As  high  threat  areas  change,  the  optimal  probability  of  success  route 
changes  and  is  now  0ii2 =(^)1'  due  to  traversing  17  low-low  arcs.  Figure  4  illustrates 
that  the  optimal  route  is  longer  yet  has  a  higher  probability  of  success  than  using 
the  same  route  shown  in  Figure  3.  If  the  same  route  is  used  as  in  Figure  3,  then 
the  probability  of  success  for  the  route  using  the  threat  map  from  Figure  3  becomes 
0i, 2=(^)12(wm)2, resulting  in  u li2  <  uj1:1. 

The  route  distance  that  0jm  represents  is  the  one-way  distance  from  the  BSB  to 
COP  %  when  on  threat  map  m.  In  determining  the  round-trip  distance,  we  assume 
that  the  threat  map  is  the  same  for  both  legs  of  the  total  route.  Using  our  previous 
example,  for  012  the  complete  round-trip  distance  from  the  BSB  to  COP  1  and  back 
to  the  BSB  is  (ay)34. 
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3.5  Problem  Formulation 


We  formulate  the  MILIRP  as  a  finite  horizon,  discrete  time  MDP.  The  state  space 
S  is: 

{0,1,...,^}  x  {0, 1, C2}  x  ...  x  {0,1,...,CF}  x  {0,1,...,F}  x  {1, 2, ...,  M}.  The 
inventory  at  COP  i  at  time  t  is  Xit  E  {1,  2,  ...Ci}.  The  number  of  CUAS  at  time  t  is 
ih  E  {0, 1, ...,  V}.  The  threat  level  of  the  AO  at  time  t  is  represented  by  threat  map 
rnt  E  {1,2,  ...,M}.  Let  St  =  (Xlt,X i2t, ...,  XFt,  vt,  mt)  E  S  be  the  state  at  time  t. 

The  action  space  ^4(s)  for  each  state  s  is  the  number  of  CUAS,  alt ,  sent  to  COP 
i  at  time  t.  Let  At  =  (an,  a2t,  apt)  £  A  (St)  be  a  specihc  action  taken  at  time  t. 
The  set  of  actions  available  at  time  t  are  governed  by  the  following  constraints 

F 

]T  alt  <  Vtl  V  t  E  {1,  2, ...,  T}  ,  (3.5) 

i= 1 

where  an  E  {0, 1,  Furthermore,  the  number  of  CLIAS  that  can  be  sent,  during 

any  period,  to  a  specihc  COP,  is  constrained  by  the  number  of  crews  or  the  number 
of  CUAS  such  that  ait  =  min  {vt,  crews}.  Each  CUAS  carries  its  capacity  G.  When 
making  a  delivery  to  COP  i  the  capacity  of  the  COP,  Ci,  cannot  be  exceeded.  There¬ 
fore,  any  potential  overage  is  not  delivered  and  returns  to  the  BSB  with  the  CUAS. 
For  each  COP  i  the  actual  amount  of  supplies  delivered  for  a  given  decision  is  dj(ait). 
The  capacity  constraint  of  each  COP  is  represented  by  Xlt  +  ^(a^)  <  Ci  for  each 
COP  i  and  all  times  t.  In  order  to  account  for  supply  consumption  during  time  t  let 
Ut  =  (uu,U2t,  ■■■,uFt)  be  the  amount  of  supplies  consumed  at  each  COP  at  time  t. 
The  capacity  constraint  now  becomes  Xlt  +  di(au)  —  ult  <  Ci  for  each  COP  i  and  all 
times  t. 

The  probability  of  a  CUAS  successfully  traversing  a  route  from  the  BSB  to  COP 
i  on  threat  map  k  is  (j)^.  When  a  single  CUAS  is  sent  to  a  COP,  there  are  three 


possible  outcomes:  1)  successfully  traversing  the  route  to  and  from  the  COP  (SS), 
2)  successfully  traversing  the  route  to  the  COP  and  failing  to  complete  the  route 
on  the  way  back  to  the  BSB  from  the  COP  (e.g.,  shot  down  by  enemy  forces  or 
suffering  mechanical  failure)  (SF),  and  3)  failing  to  traverse  the  route  while  traveling 
to  the  COP  (F).  The  probability  of  each  of  the  outcomes  is  governed  by  </>**,,  which  is 
obtained  using  Dijkstra’s  algorithm  to  find  the  path  with  the  highest  probability  of 
successful  completion  of  a  route  from  the  BSB  to  COP  i  given  threat  map  k.  Figure 
5  illustrates  the  relationship  between  the  outcomes  and  their  associated  probability 
of  occurrence.  The  outcomes  are  fixed  and  have  a  known  probability  of  occurrence 
that  is  dictated  by  the  threat  map.  The  probabilistic  transition  between  threat  maps 
is  Markovian.  The  transition  matrix  Q  is  a  square  M  x  M  matrix  that  describes  the 
probability  of  transitioning  between  threat  maps. 


SS 

(p2 . 

^  i.m 

1  CUAS 

SF 

tfh.mC1  “  0i.m) 

to  COP 

F 

(1  -  0i.«) 

Figure  5.  Probability  of  CUAS  Outcome  when  Supplying  a  COP 

When  making  a  decision  to  send  an  CUAS  to  COP  i  the  results  yt  =  (yu,  y-a-,  y-.u) 
are  governed  by  a  trinomial  distribution  /</>(y \aa,  ry), where 


Vi  i^irrv  0irre(l  0im)) 


(3.6) 


denotes  the  probabilities  of  an  SS,  SF,  or  F  event  occurring.  Let  yu,  y 2*,  and  y^i 
denote  the  number  of  SS,  SF,  and  F  events  that  occur,  respectively  when  resupplying 
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COP  i.  Consider  the  case  in  Table  1  where  ait  =  3.  The  outcome  yl  =  (2,  0, 1)  in  row 
three  has  a  probability  of  (tfm)yu  +  (1  —  (j>im)V3i  =  +  (1  —  tyim)1  of  occurring. 


%it  —  (uii  +  2/2  i)  (3-7) 

denote  the  number  of  CUAS  that  have  a  successful  delivery. 

Table  1.  Example  Outcome  Space  where  a-u  =  3 
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The  inventory  at  COP  i  at  t+1  is  related  to  the  inventory  at  time  t  in  the  following 
manner: 

Xit+i  =  max  {Xit  +  di(ait)  -  uit,  0}  ,  (3.8a) 

with  the  additional  consideration  that 

if  Xit+1  =  0,  then  Xit+1  <-  Q.  (3.8b) 

Because  backlogging  is  not  permitted,  the  inventory  cannot  be  negative.  Moreover, 
should  the  inventory  fall  to  zero,  then  we  assume  a  policy  where  a  ground  convoy  is 
sent  and  delivers  enough  inventory  to  bring  the  COP  to  full  capacity.  This  is  not  a 
desirable  occurrence  as  no  value  is  obtained  by  using  a  ground  convoy.  Furthermore, 
the  actual  amount  of  supplies  delivered  is  stochastic  and  depends  on  alt  and  rjl . 
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Delivery  only  occurs  where  zlt  and  ait  are  positive.  Therefore,  dit(ait),  the  amount  of 
supplies  successfully  delivered  is  (zu)(G). 

Similar  to  inventory,  the  number  of  CUAS  at  t  +  1  depends  on  the  realization 
of  a  particular  resupply  mission  outcome.  Let  llt  represent  the  number  of  CUAS 
destroyed  in  the  course  of  delivering  supplies  to  COP  i  at  time  f:(i.e.,  the  sum  of  SF 
and  F  occurences) 

lit  =  (y2i  +  V3i),  (3.9) 

The  number  of  CUAS  at  t  +  1  resulting  from  delivering  to  COP  i  at  time  t  is  vlt+\  = 
vu  —  ht ■  Therefore,  the  total  number  of  CUAS  at  t  +  1  is 

F 

vt+ 1 = (3.io) 

i—  1 

The  single  stage  reward  function  for  being  in  state  S  and  taking  action  A  at  time 
t  and  then  going  to  a  state  at  t  +  1  is  represented  by  rt(St,  At,  St+ 1).  The  reward 
function  is  determined  by  the  relationship: 

F 

rt(St,AuSt+1)  =  y  ^  dj(ait ),  V  t  €  {1,  2, T}  .  (3-11) 

1=1 

The  objective  function  is  to  maximize  the  expected  reward  over  all  t  G  {1,2, 

Let  Jt(St )  represent  the  expected  reward  gained  over  time  interval  {1,2,...,T}  for 
delivering  supplies  by  CUAS.  The  value  of  Jt{St )  is  calculated  by  solving  the  Bellman’s 
equations  where 

MS,)  =  XI  W|S„^(S1))[r1(Si,  .4,,  s')  +  J,+1(s')]}.  (3.12) 

s'eS 

Let  J*(s )  be  the  optimal  expected  reward  of  state  s  G  St  where  J*(St )  equals  the 
Jt,(s )  that  maximizes  Therefore,  J*(St )  solves  the  objective  function  by  the 
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following  equation: 


J*(St)  =  max  \  y^p(s\St,  At(St))[rt{St,  At,s)  +  Jf+i(s)]  }  (3.13) 

at(St)eMSt)  J 

where  s  G  St+i  =  (Xlt+1,  X2t+i, XFt+1,vt+1,mt+i). 

3.6  Turnpike  Methodology 

The  time  horizon  we  consider  in  our  formulation  spans  a  year  deployment  of  an 
BCT.  Through  the  course  of  the  deployment  the  BCT  experiences  all  four  seasons, 
each  of  which  presents  a  different  threat  condition.  We  account  for  the  differences 
between  seasons  by  considering  each  as  a  separate  instance  and  determine  the  pro¬ 
portion  of  supplies  that  can  be  delivered  by  CUAS  in  each  season. 

In  a  finite  horizon,  as  one  reaches  the  terminal  epochs,  maintaining  the  fleet  of 
CUAS  has  little  value  when  evaluating  the  decision  of  how  many  CUAS  to  use  in 
delivering  supplies  during  a  high  threat  environment.  The  reward  gained  by  delivering 
more  supplies  outweighs  the  need  to  retain  CUAS  for  future  deliveries  at  the  end  of 
the  time  horizon.  However,  reality  dictates  that  from  season  to  season  and  even  at 
the  completion  of  a  deployment,  retention  of  CUAS  has  value  and  meaning. 

We  implement  a  two-step  method  where  we  first  solve  for  the  turnpike,  the  optimal 
infinite-horizon  decision  rule,  through  value  iteration  and  then  implement  the  decision 
rule  in  the  finite-horizon  model  [21].  Let  n  be  the  set  of  optimal  policies  that  represent 
the  turnpike  and  let  n  G  n  be  a  specific  policy  when  in  state  St-  The  finite-horizon 
model  has  a  horizon,  t' ,  such  that  t'  >  t*,  where  t*  is  the  epoch  in  which  the  finite- 
horizon  model  reaches  the  turnpike  policy,  after  which  backward  induction  is  used  to 
determine  the  decisions  for  the  remaining  t'  —  t*  epochs.  At  the  first  decision  epoch, 
in  which  7 r  is  the  optimal  policy,  consider  in  the  next  epoch  that  t'  decision  epochs 
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still  remain  [21].  We  determine  the  turnpike  policy  using  an  infinite- horizon  model 
then  fix  this  policy  for  employment  in  the  finite-horizon  model,  in  which  we  are  most 
interested.  This  procedure  is  used  so  that  we  avoid  risky  CUAS  delivery  policies. 
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IV.  Computational  Example 


4.1  One  COP  Scenario 

A  general  formulation  of  the  MILIRP  is  provided  in  Section  3.5.  We  now  consider 
a  specific  instance  of  the  BCT  MILIRP  where  we  solve  for  an  e-optimal  solution  to 
supplying  a  single  battalion.  We  assume  that  the  BCT  delivers  to  a  single  COP  where 
the  battalion  headquarters  is  located.  Any  subsequent  distribution  of  supplies  by  the 
battalion  (i.e.,  supplying  the  COPs  of  subordinate  units)  is  not  considered  in  this 
problem  instance.  In  developing  a  solution  that  maximizes  the  amount  of  supplies 
delivered  by  CUAS  over  the  course  of  a  deployment,  we  discretize  the  time  horizon  and 
consider  each  season  separately.  This  enables  the  development  of  seasonal  solutions 
and  takes  advantage  of  the  link  between  the  seasons  and  the  threat  to  CUAS. 

4.2  Model  Parameterization 

We  consider  the  simple  case  of  one  battalion  sized  COP.  The  consumption  rate 
and  the  maximum  amount  of  on-hand  supplies  determines  the  inventory  capacity  of 
the  COP.  General  Dynamics  Information  Technology  [12]  indicates  projected  daily 
consumption  of  supplies  by  an  infantry  company  is  29,928.2  lbs.  An  infantry  bat¬ 
talion  consists  of  a  headquarters  company,  three  infantry  companies,  and  a  weapons 
company  [7].  The  headquarters  company  and  weapons  company  combined  are  gen¬ 
erally  the  same  size  as  an  infantry  company.  Therefore,  we  model  the  battalion  as 
having  a  daily  requirement  of  four  times  that  of  an  infantry  company.  In  order  to 
maintain  consistency  in  discretization,  we  assume  the  battalion  consumes  112,000  lbs 
of  supplies  daily.  Furthermore,  we  fix  the  storage  capacity  to  seven  days  of  supplies 
with  the  consideration  that  the  ability  to  hold  a  week’s  worth  of  supplies  provides  a 
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conservative  estimate  of  required  capacity.  Therefore,  storage  capacity  C  is  784,000 
lbs. 

We  assume  each  season  consists  of  90  days.  We  further  discretize  time  into  epochs 
that  divide  a  day  into  four  six-hour  time  periods.  This  time  resolution  enables  the 
modeling  of  day  and  night  resupply  operations.  This  is  a  valid  assumption,  as  General 
Dynamics  Information  Technology  [12]  and  the  Department  of  the  Army  [11]  both 
state  that  the  implementation  of  CUAS  enables  the  delivery  of  supplies  in  limited 
visibility  situations.  Furthermore,  all  round  trip  routes  can  be  completed  in  less  than 
six  hours  to  include  the  time  required  to  reload,  refuel,  and  conduct  preventative 
maintenance  checks  and  services  (PMCS)  between  routes.  This  also  roughly  matches 
the  average  flight  hours  associated  with  a  “Jingle  Air”  mission  of  6.4  hours  [12]. 

In  order  to  determine  the  value  to  use  for  the  capacity  of  the  CUAS,  G ,  we 
consider  the  helicopters  used  by  the  Army  and  current  CUAS  efforts  in  the  industry 
as  benchmarks.  The  sling  load  weight  of  these  helicopters  is  the  value  of  interest  as  this 
is  the  capacity  that  can  be  attached  underneath  the  helicopter  and  easily  unloaded. 
Department  of  the  Army  [5]  states  that  the  maximum  sling  load  weight  of  the  UH- 
60A  Blackhawk  as  8,000  lbs.  and  the  CH-47D  Chinook  as  26,000  lbs.  Department 
of  the  Army  [11]  identifies  4,500  lbs.  as  the  median  load  that  CUASs  should  be  able 
to  handle  from  a  design-and-capabilities  perspective  moving  forward  in  development 
and  procurement.  General  Dynamics  Information  Technology  [12]  recommends  4,000 
lbs.  as  the  load  capacity  as  it  is  an  even  multiple  of  the  standard  pallet  weights  used 
by  the  Army.  Furthermore,  the  average  load  of  a  “Jingle  Air”  helicopter  is  4,000  lbs. 
[12].  Considering  these  requirements,  results,  and  capabilities,  we  set  G  to  4,000  lbs. 
thereby  allowing  us  to  consider  its  capability  as  equal  to  civilian  contractors.  Using 
a  4,000  lbs.  capacity  for  the  CUAS  enables  scaleable  capacity  comparison  with  the 
Blackhawk  and  Chinook. 
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In  determining  the  size  of  the  CUAS  fleet,  we  consider  the  minimum  number  of 
CUAS  required  to  meet  the  daily  demand  of  the  battalion-size  COP.  The  battalion 
requires  112,000  lbs.  every  four  time  periods.  CUAS  capacity  of  4,000  lbs.  means 
that  a  fleet  of  seven  CUAS  is  required.  The  Department  of  the  Army  [11]  considers 
a  CUAS  platoon  to  consist  of  two  crews  with  a  fleet  of  four  CUAS.  We  apply  this 
as  the  basic  component  for  the  fleet  size  in  our  formulation.  Supplying  a  battalion 
requires  the  support  of  six  platoons  organized  into  two  companies.  Therefore,  our 
vehicle  fleet  has  12  crews  and  24  CUAS. 

We  consider  the  case  where  there  are  M  —  2  threat  maps.  We  use  a  scenario 
with  two  threat  maps  to  enable  the  modeling  of  a  low  threat  environment  and  a  high 
threat  environment.  The  transition  matrix  Q  is  a  2  x  2  matrix  and  has  the  form: 


Q 


a  (1  —  a) 
(1-/3) 


(4.1) 


The  parameters  a  and  ft  represent  the  probability  of  returning  to  the  same  threat  con¬ 
dition  within  each  season.  Table  2  shows  the  values  of  a  and  ft  for  each  season,  where 
a  represents  returning  to  a  low  threat  map  and  /3  a  high  threat  map.  These  values 
differ  between  seasons  to  reflect  the  changing  nature  of  threat  conditions  throughout 
a  year. 

The  exact  value  of  (j>im,  the  probability  of  successfully  traversing  a  route  one-way,  is 
not  parametrized.  Instead,  we  parameterize  the  probability  of  successfully  traversing 
the  tessellations  within  a  route  as:  ay  =  0.999,  u)m  =  0.950,  and  ay,  =  0.900.  These 
values  are  used  in  Equation  3.4  to  solve  for  the  optimal  route  r*  that  specifies  the 
value  of  We  parameterize  each  uj  to  generate  (f)vm  similar  to  that  used  in  the  study 
by  General  Dynamics  Information  Technology  [12],  in  which  CUAS  have  similar  loss 
rates  as  Shadow  UAS:  approximately  0.022%. 
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Table  2.  a  and  (3  for  each  Season 


Season 

a 

P 

Winter 

0.95 

0.25 

Spring 

0.85 

0.30 

Summer 

0.65 

0.90 

Fall 

0.80 

0.50 

Let  JjH(St)  be  the  optimal  value  of  being  in  state  St  under  steady-state  conditions. 
The  infinite-horizon  model  calculates  by  solving 

JiH{St )  =  max 
aeA(St) 

The  process  of  solving  JjH(St )  identifies  a  specific  action  A't(s)  to  take  when  in  state 
s.  Let  tt*ih  =  A't(s )  for  each  s,  be  an  element  of  II,  the  set  of  policies.  Let  7  be  the 
discount  factor.  Powell  [20]  offers  0  <  7  <  0.7  as  an  initial  range  to  use  for  7  in  order 
to  improve  performance.  We  let  7  =  0.9.  We  implement  a  two  tier  stopping  rule. 
First,  we  utilize  the  convergence  test 


i7I 


t= 0 


(4.2) 


=  (4.3) 

z  *  7 

where  we  set  th  tolerance  to  be  e  =  0.1.  Second,  an  upper  bound  of  N  —  100,  is 
placed  on  the  number  of  iterations  to  calculate  J(St).  LIpon  reaching  the  stopping 
criteria,  we  employ  a  backward  induction  algorithm  with  a  fixed  policy  n*IH  from 
Equation  4.2  to  solve  the  finite-horizon  problem  modeled  in  Equation  3.13. 

4.3  Base  Case  Results 

The  results  of  our  battalion-sized  one  COP  formulation  are  presented  in  Table  3. 
For  the  winter  and  spring  seasons,  a  fleet  of  24  CUAS  with  12  crews  successfully 
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delivers  about  30%  of  the  total  demand.  During  the  summer  season,  a  fleet  of  CUAS 
only  delivers  5.32%  of  the  daily  supply  requirements.  The  reason  for  the  low  percent¬ 
age  of  supplies  delivered  is  the  high  probability  of  being  in  a  high  threat  map  and 
the  corresponding  low  probability  of  successfully  completing  a  route  from  the  BSB 
to  the  COP.  The  0jm  for  the  high  threat  map  m  of  each  of  these  seasons  is  greater 
than  the  <J)im  of  the  low  threat  map  for  the  summer  season.  Therefore,  delivery  of 
supplies  during  the  summer  will  always  underperform  the  rest  of  the  seasons,  given 
the  same  initial  conditions.  During  the  fall,  about  20%  of  the  total  demand  can  be 
met  by  CUAS.  Let  S $  be  the  difference  in  the  values  of  (j)im  that  represent  the  low  and 
high  threat  conditions  within  each  season.  The  5<p  for  winter  and  spring  is  less  than 
0.003,  while  for  summer  and  fall  the  value  is  approximately  0.1.  This  reflects  that 
the  conditions  in  the  spring  and  fall  seasons  mirror  the  conditions  of  their  respective 
preceding  season.  The  steep  changes  from  fall  to  winter  and  spring  to  summer  rep¬ 
resent  how  the  nature  of  the  threat  condition  changes  abruptly  during  these  times  of 
the  year.  Furthermore,  the  large  S $  for  the  summer  and  fall  seasons  represents  a  very 
low  probability  of  successfully  delivering  supplies  to  the  COP. 

Table  3.  Percent  of  Supplies  Delivered  by  Season:  Base  Case 


Season 

01,1 

01,2 

%  Supplies  Delivered 

Winter 

0.9860 

0.9830 

31.32 

Spring 

0.9850 

0.9830 

29.06 

Summer 

0.9800 

0.8834 

5.32 

Fall 

0.9830 

0.8854 

19.92 

The  winter  and  spring  seasons  have  the  same  optimal  policy.  Within  these  seasons 
the  optimal  policy  does  differ  for  low  and  high  threat  days.  When  there  are  zero  to 
seven  CUAS,  send  all  CUAS  available.  When  eight  to  24  CUAS  are  in  the  fleet, 
send  enough  CUAS  to  meet  the  period  demand  regardless  of  inventory  level.  When 
a  shortage  exists,  send  all  available  np  to  reaching  the  point  where  the  COP  capacity 
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is  met.  Then  send  only  enough  CUAS  to  bring  the  COP  to  capacity.  This  policy 
likely  results  from  the  high  probability  of  being  in  a  low  threat  map  and  the  small 
difference  in  </>jm  between  the  low  threat  and  high  threat  maps. 

The  summer  and  fall  seasons  have  the  same  optimal  policy  as  the  winter  and  spring 
when  in  the  low  threat  map  condition.  During  the  summer,  when  in  the  high  threat 
map,  all  CUAS  are  sent  without  delivering  overage  due  to  the  capacity  constraint. 
This  occurs  because  the  (f)im  of  the  high  threat  map  is  not  so  low  as  to  overly  restrict 
sending  CUAS  to  make  deliveries  considering  the  situation  at  the  next  time  period 
will  not  likely  improve  given  the  high  value  of  f3.  During  the  fall,  when  there  are  seven 
or  fewer  CUAS,  no  deliveries  are  attempted.  When  there  are  more  than  eight,  as  the 
number  of  CUAS  increase  then  more  are  sent  to  create  an  inventory  that  prevents 
stock-out.  As  the  number  of  CUAS  approachs  24  then  deliveries  are  made  to  bring 
the  inventory  up  to  capacity. 

4.4  Sensitivity  Analysis  on  a,  /3  and  Crews 

The  parameterization  of  a  and  /3  represents  an  assessment  of  the  nature  of  the 
transition  between  threat  environments  within  a  season.  However,  these  values  are 
dynamic  and  are  dictated  by  the  activities  of  hostile  forces.  We  conduct  a  sensitivity 
analysis  of  these  parameters  to  gain  insight  on  the  nature  of  their  influence  on  the 
ability  to  deliver  supplies  in  a  combat  environment.  Table  4  contains  the  results  of 
the  initial  assessments  of  a  and  f3  as  well  as  two  additional  parameterizations  for  each 
season.  We  consider  the  initial  parameters  to  be  the  best  representation  of  the  values 
and  an  upper  limit  on  the  nature  of  the  relationship  between  a  and  f3.  The  other 
two  parameterizations  represent  a  lower-limit  representation  of  the  relationship  and 
an  assessment  in  the  middle,  respectively. 
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Table  4.  Percent  of  Supplies  Delivered  by  Season:  Varying  a  and  (3 


Season 

a 

P 

0i,i 

01,2 

%  Supplies  Delivered 

0.95 

0.25 

0.9860 

0.9830 

31.32 

Winter 

0.90 

0.30 

0.9860 

0.9830 

30.91 

0.85 

0.35 

0.9860 

0.9830 

30.52 

0.85 

0.30 

0.9850 

0.9830 

29.06 

Spring 

0.80 

0.35 

0.9850 

0.9830 

28.84 

0.75 

0.40 

0.9850 

0.9830 

28.63 

0.65 

0.90 

0.9800 

0.8834 

5.32 

Summer 

0.70 

0.85 

0.9800 

0.8834 

5.99 

0.75 

0.80 

0.9800 

0.8834 

6.85 

0.80 

0.50 

0.9830 

0.8854 

19.92 

Fall 

0.75 

0.55 

0.9830 

0.8854 

19.07 

0.70 

0.60 

0.9830 

0.8854 

17.39 

During  the  winter  and  spring  season,  regardless  of  the  values  for  a  and  /?,  a  fleet 
of  CUAS  still  delivers  around  30%  of  the  supplies  required.  The  percent  of  supplies 
delivered  by  CUAS  does  decrease  some,  but  the  difference  between  the  maximum  and 
minimum  amounts  is  only  1%.  We  investigate  the  optimal  policy  for  the  winter  and 
spring  seasons  to  determine  why  the  percentage  of  supplies  delivered  by  CUAS  does 
not  significantly  change.  In  general,  when  more  CUAS  exist  than  crews,  the  optimal 
policy  follows  a  structure  where,  as  inventory  increases  the  number  of  CUAS  sent  to 
deliver  supplies  decreases  from  maximum  number  of  crews  available  to  enough  CUAS 
to  bring  the  inventory  up  to  the  storage  capacity  without  delivering  an  overage  of 
supplies.  During  the  winter  and  spring  seasons  the  difference  between  a  and  (3  ranges 
between  0.35  and  0.70  with  a  always  greater  than  (3.  This  large  difference  indicates 
that  the  high  probability  of  recurrence  of  a  low  threat  map  and  low  probability  of 
recurrence  of  a  high  threat  map,  coupled  with  the  high  (pim  of  the  high  threat  map, 
makes  dilivering  supplies  advantageous  even  on  a  high  threat  map.  The  high  threat 
map  occurs  infrequently  enough  that  sending  CUAS  on  those  days  is  worth  the  risk  of 
being  destroyed  by  enemy  action.  The  high  values  of  tpim  make  successful  delivery  of 
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supplies  very  certain;  therefore,  even  as  a  decreases  and  f3  increases,  overall  delivery 
is  not  affected  very  much.  This  indicates  that,  for  high  values  of  the  probability 
of  remaining  in  a  threat  map  is  low  enough  that  it  does  not  significantly  affect  the 
percent  of  supplies  delivered  by  CUAS. 

The  summer  and  fall  seasons  experience  similar  change  in  the  delivery  of  supplies. 
During  these  seasons,  as  a  increases  and  f3  decreases,  1%  more  supplies  are  delivered. 
The  optimal  policy  on  the  low  threat  map  is  the  same  policy  as  that  used  during 
the  winter  and  spring  seasons.  During  high  threat  map  conditions,  summer  delivery 
policy  remains  the  same  as  in  low  threat  map  conditions:  send  CUAS  to  supply  up 
to  COP  capacity.  The  high  threat  condition  optimal  policy  for  the  fall  season  differs 
from  the  summer’s.  In  the  fall,  when  there  are  eight  CUAS  or  more,  supplies  are 
delivered  such  that,  as  the  number  of  CUAS  increases,  more  deliveries  are  made  even 
when  inventory  approaches  capacity.  When  the  number  of  CUAS  are  close  to  eight, 
then  CUAS  are  sent  only  until  inventory  has  a  small  buffer  to  prevent  stock-out. 
As  the  number  of  CUAS  increases,  so  do  the  number  of  deliveries  even  as  inventory 
increases  toward  capacity.  When  there  are  seven  or  less  CUAS,  then  no  CUAS  are 
sent  to  deliver  supplies.  The  difference  betwen  a  and  f3  during  the  summer  and  fall 
seasons  ranges  between  0.10  and  0.30  where,  during  the  summer,  f3  is  always  greater 
than  a,  and  in  the  fall  the  reverse  is  true.  The  probability  of  remaining  in  the  current 
threat  map  is  at  least  0.50  in  both  seasons.  This  condition  and  the  low  (f)im  of  the 
high  threat  map  indicate  that  the  value  and  relationship  of  a  and  (3  significantly 
affect  the  percentage  of  supplies  delivered  by  CUAS.  When  f3  is  great  than  a ,  low 
delivery  occurs,  as  in  the  summer.  When  a  is  greater  than  (3,  then  even  with  a  low 
4>im,  successful  delivery  of  supplies  approaches  that  of  when  4>im  is  high.  The  higher 
percent  of  supplies  delivered  by  CUAS  in  the  fall  being  close  to  that  of  winter  and 
spring  is  evidence  of  this  case. 
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We  also  consider  the  case  where  the  BSB  has  the  ability  to  surge  and  has  more  than 
12  crews  available  to  send  more  CUAS  during  a  single  time  period  in  order  to  deliver 
supplies.  Given  the  structure  of  the  units  we  consider  the  cases  where  16  and  18  crews 
are  available.  Table  5  shows  the  results  of  this  analysis,  given  the  a  and  (3  values 
used  in  Table  3.  The  results  show  that,  no  matter  the  number  of  additional  crews 
available,  the  percent  of  supplies  delivered  by  CUAS  does  not  change.  This  indicates 
that  ( pim ,  a,  and  /3  have  a  stronger  influence  on  the  ability  to  deliver  supplies  than 
the  number  of  crews.  As  previously  stated,  as  long  as  the  number  of  CUAS  meets  or 
exceeds  seven,  then  the  daily  supply  requirement  can  be  met  and  stock-out  avoided. 

Table  5.  Percent  of  Supplies  Delivered  by  Season:  Varying  crew 


Season 

crew 

a 

P 

%  Supplies  Delivered 

Winter 

12 

0.95 

0.25 

31.32 

16 

0.95 

0.25 

31.32 

18 

0.95 

0.25 

31.32 

Spring 

12 

0.85 

0.30 

29.06 

16 

0.85 

0.30 

29.06 

18 

0.85 

0.30 

29.06 

Summer 

12 

0.65 

0.90 

5.32 

16 

0.65 

0.90 

5.32 

18 

0.65 

0.90 

5.32 

Fall 

12 

0.80 

0.50 

19.92 

16 

0.80 

0.50 

20.85 

18 

0.80 

0.50 

20.85 

4.5  Design  of  Experiment 
4.5.1  Design. 

The  case  presented  in  Section  4.1  represents  a  particular  instance  of  the  MILIRP. 
To  understand  the  influences  on  the  percent  of  supplies  delivered  by  CUAS  we  develop 
a  design  of  experiment.  We  use  a  central  composite  design  consisting  of  a  fractional- 
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factorial  design,  a  center  run,  and  axial  runs.  For  the  fractional-factorial  we  implement 
a  one-half  fraction  2k~p  resolution  VI  design  where  k  is  the  number  of  factors  that  we 
believe  significantly  effect  the  response,  the  percent  of  supplies  delivered  by  CUAS, 
and  p  is  the  number  of  independent  effects  to  be  confounded,  and  two  signifies  the 
number  of  levels  used  to  analyze  each  factor.  A  resolution  R  design  is  a  design  where 
no  p-factor  effect  is  aliased  with  another  effect  consisting  of  less  than  R  —  p  factors 
[17].  For  this  design,  we  use  a  design  model  with  k  —  6  factors  and  p  —  1  with 
each  factor  having  two  settings:  high  and  low.  Furthermore,  the  design  is  resolution 
VI  because  the  minimum  length  alias  effect  of  a  one-factor  effect  has  five  factors, 
whereby  R  =  6.  The  factors  we  consider  to  have  a  significant  effect  on  the  proportion 
of  supplies  delivered  are:  1)  0/,  the  threat  level  of  the  low  threat  map,  2)  6$,  the 
difference  between  the  threat  level  of  the  low  threat  map  and  high  threat  map,  3) 
V,  the  total  number  of  CUAS,  4)  a,  the  one-step  probability  of  returning  to  the  low 
threat  map,  5)  (5,  the  one-step  probability  of  returning  to  the  high  threat  map,  and  6) 
the  number  of  crew  that  control  the  CUAS,  where  one  crew  operates  one  CUAS.  The 
high  and  low  settings  of  the  factors  as  well  as  the  center-run  and  axial  values  used 
appear  in  Table  6.  The  axial  and  center- runs  allow  us  to  test  for  curvature  in  the 
response  to  determine  if  a  polynomial  regression  describes  the  percentage  of  supplies 
delivered  by  CUAS  better  than  a  linear  regression.  The  axial-run  modifies  a  center 
run  with  an  extension  in  one  direction  for  a  single  factor.  Axial  runs  for  each  extreme 
of  each  factor  are  conducted.  We  include  only  one  center-run  because  our  Markov 
decision  process  is  deterministic.  Therefore,  the  response  from  multiple  center-runs 
will  be  the  same  value  and  will  not  provide  any  additional  indication  of  curvature 
than  measurable  from  a  single  run. 

We  base  the  parameterization  of  our  factor  settings  on  reports  and  studies  spon¬ 
sored  by  the  Army  and  on  the  conditions  we  desire  to  model.  General  Dynamics 
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Table  6.  Factor  Settings  for  2 y72  One-Quarter  Fractional  Factorial  Design 


Factor 

-1.7244 

-1 

0 

1 

1.7244 

<t>i 

0.9810 

0.9848 

0.9900 

0.9952 

0.9990 

8<t> 

0.016 

0.03 

0.05 

0.07 

0.084 

a 

0.51 

0.61 

0.75 

0.89 

0.99 

P 

0.02 

0.22 

0.5 

0.78 

0.98 

crew 

0.39U 

0.48U 

0.60U 

0.72U 

0.81V 

V 

8 

15 

20 

25 

28 

Information  Technology  [12]  conducts  a  business  analysis  of  different  scenarios  with 
CUAS  and  other  modes  of  supply  delivery.  This  analysis  considers  the  CUAS  to 
behave  similar  to  the  Shadow  UAS  with  aircraft  loss  rates  of  0.022%  and  weather 
related  impacts  on  mission  of  3.4%.  Using  this  data  to  bound  our  0;  and  5$,  we 
set  the  upper  bound  on  <pi  based  on  the  aircraft  loss  rate  and  set  the  lower  bound 
such  that  the  lower  bound  of  (f>i  and  lower  bound  on  8 $  approximate  the  affect  of 
weather.  The  lower  bound  of  5 $  also  allows  us  to  model  the  condition  where  there 
is  little  change  in  the  overall  threat  level  after  the  transition  governed  by  Q.  The 
upper  bound  on  8$  allows  us  to  model  high  threat  environment,  where  0.91  is  the 
probability  of  successfully  traversing  a  route  to  the  COP.  The  high  and  low  levels  we 
use  for  a  and  (3  enable  the  modeling  of  a  wide  range  of  conditions.  The  upper  bound 
of  0.89  for  a  models  the  condition  where  a  return  to  the  current  threat  condition  is 
very  likely.  Similarly,  using  0.61  indicates  a  moderately  low  probability  of  staying 
in  the  current  threat  condition.  The  value  of  0.78  for  the  high  level  of  f3  represents 
a  moderately  high  likelihood  of  returning  to  a  high  threat  environment.  The  low 
value  of  0.22  for  f3  reprents  a  infrequent  recurrence  to  a  high  threat  environment. 
The  high  and  low  setting  values  for  a  and  (3  provide  a  buffer  for  the  axial  runs  to 
test  the  extreme  cases  for  each  factor  while  maintaining  useful  parameter  values  that 
have  meaning  for  application.  The  value  for  the  low  factor  setting  of  V  comes  from 
adjusting  to  allow  the  lower  axial  value  to  maintain  the  minimum  required  CUAS 


44 


to  satisfy  1%.  The  upper  bounds  for  these  factors  come  from  the  Department  of  the 
Army  [11],  where  a  platoon  of  CUAS  consists  of  four  aircraft  and  two  crews.  We 
consider  the  case  where  two  companies  of  three  platoons  each  exist  with  the  mission 
to  support  the  supply  needs  of  an  infantry  battalion.  We  set  the  high  level  number 
of  CUAS  as  25  to  bracket  this  configuration.  We  consider  crew  as  a  proportion  of 
V,  which  implies  it  is  a  continuous  factor.  However,  only  whole  crews  are  feasible; 
therefore  we  round  the  value  of  crew  when  using  it  in  the  model. 

The  central  composite  design  we  implement  contains  a  one-half  fraction  design 
from  Montgomery  [17]  that  uses  32  runs  for  the  entire  analysis  instead  of  64  runs 
required  for  a  full  26  factorial  design.  The  fractional  design  provides  a  more  efficient 
use  of  runs  and  allows  for  the  estimation  of  the  factor  effects.  Furthermore,  we  conduct 
12  axial-runs  with  a  normalized  distance  of  1.7244  units  from  the  center  value  for  each 
factor  and  a  single  center-run.  The  entire  central  composite  design  uses  45  runs  to 
test  for  the  presence  of  curvature  in  the  response,  which  indicates  a  second-order  or 
higher  polynomial  model.  The  central  composite  design  we  implement  appears  in 
Table  7.  The  design  in  Table  7  is  completely  orthogonal.  Therefore,  if  one  or  several 
of  the  design  factors  do  not  significantly  affect  the  percentage  of  supplies  delivered 
by  CUAS,  then  removing  those  factors  from  the  model  will  not  affect  the  estimated 
effects  of  the  other  factors. 

4.5.2  Results  and  Analysis. 

We  utilize  the  results  of  the  design  of  experiment  in  Subsection  4.5.1  to  conduct 
a  factor  screening  experiment.  The  design  enables  a  total  of  21  main  factors  and 
two-factor  interactions  effects  and  six  quadratic  effects  to  be  estimated.  The  screen¬ 
ing  experiment  identifies  which  of  these  factors  significantly  influence  the  percent  of 
supplies  delivered  by  CUAS.  In  this  analysis  we  consider  a  P-value  of  <  0.05  as  sig- 
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nificant.  The  following  seven  factors  significantly  influence  the  response:  (pi,  V,  a,  f3 , 
8$,  (pi  *  a,  (pi  *  (3,  a  *  (3,  a  *  a,  and  8$  *  8$.  The  sixth,  seventh,  and  eighth  factors 
are  two-factor  interactions  and  the  last  two  are  quadratic  factors.  The  exclusion  of 
crew  in  the  model  raises  an  interesting  question.  Inherently,  as  the  number  of  crew 
increases  so  does  the  number  of  CUAS  available  to  deliver  supplies  at  any  given  time; 
however,  results  indicate  that  crew  does  not  impact  the  percent  of  supplies  delivered 
by  CUAS  significantly.  The  exclusion  of  crew  from  the  model  begins  to  make  sense 
given  that  for  fixed  values  of  a  and  (3,  changing  crew  did  not  change  the  percent¬ 
age  of  supplies  delivered  by  CUAS.  Table  8  summarizes  the  results  of  the  screening 
experiment  and  regression  model  generated  by  the  factors  included  in  the  model. 

Table  8.  Factor  Influence  on  Percent  of  Supplies  Delivered  by  CUAS 


Factor 

Sum  of  Squares 

%  Contribution 

P -Value 

a 

1309.5966 

22.2049 

<  0.0001 

f3 

1171.8885 

19.8700 

<  0.0001 

(pi 

1046.0373 

17.7361 

<  0.0001 

V 

652.3636 

11.0611 

<  0.0001 

a  *  (pi 

286.2028 

4.8527 

0.0002 

/3*(pi 

266.6895 

4.5219 

0.0004 

a*  f3 

202.8098 

3.4387 

0.0015 

5<p  *  8 ^ 

177.0541 

3.0020 

0.0027 

8 4 , 

114.0517 

1.9338 

0.0139 

a  *  a 

95.3271 

1.61632 

0.0235 

Table  8  provides  several  interesting  results.  First,  the  main  factors  account  for 
72.8%  of  the  error  in  the  model.  This  indicates  that  the  factors  a,  f3,  (pi ,  and  V  exhibit 
the  most  influence  over  the  actual  percentage  of  supplies  delivered  by  CUAS  during  a 
season.  The  two-factor  interactions  explain  12.8%  of  the  error  with  quadratic  effects 
explaining  only  4.6%.  These  are  the  combinations  of  factors  used  to  describe  the 
threat  environment  and  indicates  that  the  inter-dynamics  of  the  environment  is  not 
as  important  as  the  individual  components.  The  low  quadratic  factor  explanation  of 
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error  represents  small  curvature  in  the  percent  of  supplies  delivered  by  CUAS  due  to 
compounding  a  single  factor’s  effect. 

Table  9.  Effects  Model  for  Percent  of  Supplies  Delivered  by  CUAS 


Summary  of  Fit 

R2 

0.9024 

Adjusted  K2 

0.8737 

Parameter  Estimates 

Term 

Estimate 

Lower  95% 

Upper  95% 

a 

5.8746 

4.5170 

7.2322 

P 

-5.5572 

-6.9148 

-4.1995 

<!>i 

5.2503 

3.8927 

6.6079 

V 

4.1462 

2.7886 

5.5038 

3<j>  *  b(j> 

3.1641 

1.1754 

5.1527 

Oi  *  <pi 

2.9906 

1.5112 

4.4690 

(3  *4>i 

-2.8869 

-4.3653 

-1.4085 

a  *  /3 

-2.5175 

-3.996 

-1.0391 

a  *  a 

2.3217 

0.3330 

4.3103 

S<t> 

-1.7336 

-3.0912 

-0.3760 

The  model  we  develop  using  the  factors  previously  discussed  provides  a  quadratic 
regression  model  with  the  parameters  found  in  Table  9.  This  model  explains  90.24% 
of  the  error  in  the  percent  of  supplies  delivered  by  CUAS.  Furthermore,  the  difference 
in  the  R2  and  Adjusted  R 2  values  is  small.  This  indicates  that  only  useful  factors  are 
in  the  model  [15]. 

We  develop  the  regression  model  not  as  a  prediction  model  but  to  gain  insight  into 
the  factor  effects  on  the  percentage  of  supplies  delivered  by  CUAS.  The  estimates  for 
each  factor  are  the  coefficients  in  the  regression  model  and  indicate  the  change  in  the 
response  over  the  range  of  the  factor.  Since  each  factor  has  two  levels,  the  effect  of 
the  factor  is  twice  the  coefficient  estimate. 

The  most  significant  factor,  a,  represents  an  11%  increase  in  the  ability  to  provide 
supplies  by  CUAS.  As  the  probability  of  remaining  in  a  low  threat  environment 
increases,  there  are  more  low  threat  periods  enabling  more  successful  deliveries  of 
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supplies  from  the  BSB  to  the  COP.  Likewise,  f3  has  a  negative  effect  on  the  percentage 
of  supplies  delivered  by  CUAS  of  11%.  When  the  probability  of  remaining  in  a  high 
threat  condition  increases,  the  ability  to  deliver  supplies  decreases  as  the  probabiliy 
of  successfully  traversing  a  route  between  the  BSB  and  COP  deminishes.  While 
these  factors  offset  the  other’s  effect  in  magnitude,  a  and  f3  have  an  interaction 
affect  that  has  a  negative  effect  of  5%  on  the  percent  of  supplies  delivered  by  CUAS. 
This  means  that  as  a  and  (3  increase,  the  percentage  of  supplies  delivered  by  CUAS 
decreases.  This  shows  that  even  when  the  probability  of  returning  to  a  low  threat 
condition  increases,  the  increase  in  probability  of  returning  to  a  high  threat  condition 
has  a  dominant  effect  on  the  percentage  of  supplies  delivered.  However,  a  also  has  a 
quadratic  effect  that  has  a  4.5%  increase  in  the  percent  of  supplies  delivered  by  CUAS 
over  the  range  given  in  Table  6.  This  accounts  for  the  compounding  effect  of  having 
continually  low  threat  conditions.  These  inter-relations  of  a  and  f3  demonstrate  the 
importance  of  correctly  identifying  the  threat  condition  and  how  likely  it  is  to  change. 
When  transitioning  between  low  and  high  threat  conditions,  as  long  as  the  probability 
of  staying  in  the  say  threat  condition  is  about  equal,  then  the  percentage  of  supplies 
dcliveredy  by  CUAS  will  be  above  20%  as  evident  during  the  fall  season  in  Table  4. 

The  threat  level  of  the  low  threat  map  <pi  has  a  10%  positive  effect  on  the  per¬ 
centage  of  supplies  delivered  by  CUAS  over  the  range  of  the  factor.  This  means  that, 
as  the  probability  of  successfully  traversing  a  route  from  the  BSB  to  COP  increases, 
so  does  the  percentage  of  supplies  delivered  by  CUAS.  The  factor  (pi  also  interacts 
with  both  a  and  (3.  The  a  *  <pi  two-factor  interaction  has  an  effect  of  6%  over  the 
range  of  the  interaction.  Therefore,  for  a  fixed  (pi,  increasing  the  probability  of  stay¬ 
ing  in  a  low  threat  environment  increases  the  effect  of  successfully  traversing  a  route. 
Similarly,  the  (3  *  (pi  two-factor  interaction  has  a  -6%  effect.  When  (pi  is  fixed,  as 
the  probability  of  staying  in  a  high  threat  environment  increases  the  percentage  of 
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supplies  delivered  by  CUAS  decreases.  Therefore,  the  net  impact  of  the  interactions 
for  a  fixed  (pi,  when  considering  all  other  factors  in  the  model,  is  zero.  This  explains 
the  results  for  all  seasons  from  Section  4.3.  While  the  <pim  varied,  the  values  of  a 
and  f3  really  determined  the  amount  of  supplies  delivered.  This  further  explains  why 
deliveries  were  attempted  in  the  summer  even  when  conditions  were  bad.  When  in  a 
high  threat  map,  the  likelihood  of  conditions  improving  is  so  low  and,  even  when  a 
low  threat  condition  is  reached,  it  will  not  remain.  Therefore,  the  decision  to  accept 
the  risk  of  sending  on  a  high  threat  map  is  taken,  resulting  in  low  percent  of  supplies 
delivered. 

The  CUAS  fleet  size  V  affects  the  percent  of  supplies  delivered  by  8%.  This 
represents  the  effect  that  the  Army  has  on  the  percentage  of  supplies  delivered  by 
CUAS.  All  other  factors  are  exogenous  and  are  elements  of  the  operating  environment. 
The  ability  to  deliver  supplies  does  not  depend  on  crew  but  on  the  total  number  of 
available  CUAS.  If  spare  CUAS  exist  then  no  matter  the  value  of  crew,  delivery  is 
possible.  If  V  is  less  than  crew,  then  regardless  of  crew  only  the  number  of  V  CUAS 
are  available  to  send. 

The  final  factor  in  the  model  is  S The  increasing  of  decreases  the  percentage 
of  supplies  delivered  by  CUAS  by  3%.  This  means,  as  the  difference  in  probability  of 
successfully  traversing  a  route  between  low  and  high  threat  conditions  increases,  the 
percent  of  delivered  supplies  decreases.  The  factor  S $  also  has  a  positive  quadratic 
effect  of  6%.  This  occurs  because,  as  S $  increases  the  probability  of  sending  a  CUAS 
on  the  higher  threat  day  decreases  and  sending  a  CUAS  on  a  low  threat  day  becomes 
more  of  a  certainty.  Because  of  the  higher  probability  of  an  SS  occurance  on  a  low 
threat  day,  the  likelihood  of  success  increases  and  a  maximum  number  of  CUAS 
are  sent  when  favorable  conditions  exist.  Furthermore,  the  quadratic  effect  of  8phi 
represents  sequential  days  where  the  high  threat  condition  is  much  lower  than  the 
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low  threat  condition.  When  this  occurs,  the  likelihood  of  a  low  threat  environment 
decreases  and  the  risk  of  losing  a  CUAS  in  a  high  threat  conditions  is  accepted  in 
order  to  maintain  inventory  at  the  COP  and  prevent  stock-out. 
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V.  Conclusions  and  Recommendations 


5.1  Conclusion 

The  deterministic  demand  MILIRP  problem  formulation  we  present  provides  Army 
decision-makers  a  model  for  analyzing  the  requirements  of  a  CUAS  fleet  dedicated 
to  sustainment  operations.  We  demonstrate  that  a  BSB  that  has  two  CUAS  compa¬ 
nies  having  a  combined  fleet  of  24  CUAS  and  12  crew  has  the  ability  to  resupply  an 
infantry  battalion.  This  means  that  an  entire  IBCT  can  be  supplied  by  six  CUAS 
companies.  Even  if  complete  resupply  by  CUAS  is  not  possible  due  to  extreme  threat 
conditions,  as  in  the  summer  and  fall  seasons,  the  amount  of  supplies  delivered  by 
ground  convoy  or  manned  aviation  assets  can  be  reduced  by  20%.  This  significantly 
reduces  soldier  exposure  to  enemy  attack,  increases  reliability  in  supply  delivery,  and 
allows  more  manned  aviation  assets  to  fill  other  operational  mission  requirements. 
Our  formulation  of  the  MILIRP  provides  a  framework  for  modeling  the  capability  of 
resupply  by  CUAS.  In  particular,  properly  assessing  the  probability  of  completing  a 
route  in  varying  threat  conditions  and  the  probability  of  staying  in  and  transitioning 
to  different  threat  conditions  is  key  to  modeling  the  use  of  CUAS  in  conducting  re¬ 
supply.  This  model  can  be  used  to  evaluate  the  proposed  results  of  other  studies  (i.e. , 
General  Dynamics  Information  Technology  [12])  and  the  present  results  with  trans¬ 
parency.  The  dynamic  programming  formulation  provides  clear  parameterization  and 
allows  for  easy  assessment  of  the  results. 

5.2  Limitations 

The  current  formulation  and  application  only  models  a  single  battalion-sized  COP. 
The  requirements  to  supply  an  entire  BCT  can  be  modeled  as  a  multiple  of  the  results 
for  a  battalion.  However,  this  requires  extrapolation  of  the  results  and  an  increase  in 
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the  variance  regarding  any  estimates  in  CUAS  fleet  size  requirements.  Furthermore, 
the  design  of  experiment  provides  insight  into  the  significant  factors  and  their  effects; 
however,  selecting  a  specific  range  for  the  values  to  use  in  parameterizing  the  different 
factors  results  in  only  a  portion  of  the  entire  design  space  being  investigated  by  our 
analysis.  Therefore,  the  percent  of  supplies  delivered  by  CUAS  and  insights  on  the 
significant  factors  only  apply  under  the  range  of  conditions  modeled. 

5.3  Potential  Future  Research 

We  consider  the  case  of  deterministic  demand  in  our  formulation  of  the  MILIRP. 
This  accounts  for  the  normal  daily  consumption  of  supplies  that  does  not  change. 
However,  in  conditions  of  increased  attacks  by  hostile  forces,  an  increase  of  supply 
consumption  occurs.  Further  analysis  of  the  MILIRP  should  consider  a  stochastic 
demand  in  addition  to  the  deterministic  demand  we  model.  We  believe  a  stochastic 
demand  that  is  conditioned  on  the  threat  condition  of  the  AO  surrounding  the  COP 
is  an  appropriate  approach  to  modeling  this  variant  of  the  MILIRP. 

Another  area  of  further  consideration  is  the  treatment  of  supplies.  In  our  formu¬ 
lation  we  consider  the  supplies  as  a  whole  and  analyze  the  total  weight  that  must  be 
delivered  to  the  COP.  However,  the  military  considers  different  categories,  classes  of 
supplies.  Each  class  of  supply  has  a  different  utilization  rate,  especially  when  consid¬ 
ering  stochastic  demand.  For  example,  the  ammunition,  water,  and  medical  supply 
consumptions  change  depending  on  the  threat  condition  and  combat  operations  the 
unit  conducts.  Therefore,  modeling  the  demand  of  each  class  of  supply  is  another 
area  that  requires  attention  and  furhter  resolution  in  modeling. 

We  apply  the  dynamic  programming  model  to  a  single  battalion  within  the  BCT. 
An  extension  of  the  model  needs  to  consider  multiple  battalions.  We  believe  the 
dynamic  programming  model  we  formulate  can  solve  the  multiple  battalion  model 
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exactly.  Kleywegt  et  al.  [14]  state  that  a  dynamic  programming  model  can  solve 
exactly  instances  that  have  three  to  four  customers,  the  number  of  battalions  of 
interest  in  the  MILIRP.  After  extending  the  model  to  include  multiple  battalion- 
sized  COPs,  separating  the  battalions  into  the  subordinate  elements  should  be  done 
until  the  model  cannot  solve  the  problem.  At  this  point,  an  approximate  dynamic 
programming  formulation  should  be  implemented  to  solve  an  instance  of  the  MILIPR 
with  up  to  25  COPs. 

Finally,  separate  analysis  is  required  to  determine  the  actual  parameterization 
of  the  significant  factors  in  the  design  of  experiment.  This  will  develop  the  actual 
response  region  of  interest  and  enable  accurate  analysis  of  the  requirements  to  com¬ 
pletely  conduct  resupply  for  an  BCT  by  CUAS  only.  Parameters  that  require  special 
attention  and  analysis  are  a,  /3,  ay,  a ;m,  and  ay,.  These  parameters  address  the 
probability  of  staying  in  a  low  and  high  threat  environment  and  the  probability  of 
successfully  traversing  a  portion  of  the  route.  These  parameters  are  exogenous  to 
what  the  Army  can  control  and  have  the  greatest  effect  on  the  ability  to  deliver 
supplies  successfully. 


54 


Bibliography 


[1]  Bertazzi,  Luca,  Bosco,  Adarno,  Guerriero,  Francesca,  &  Lagana,  Demetrio.  2013. 
A  stochastic  Inventory  Routing  Problem  with  Stock-out.  Transportation  Research 
Part  C ,  27,  89-107. 

[2]  Chien,  T.  William,  Balakrishnan,  Anantaram,  &  Wong,  Richard  T.  1989.  An 
Integrated  Inventory  Allocation  and  Vehicle  Routing  Problem.  (English).  Trans¬ 
portation  Science,  23(2),  67-76. 

[3]  Coelho,  Leandro  C.,  &  Laporte,  Gilbert.  2013.  The  Exact  Solution  of  Several 
Classes  of  Inventory-Routing  Problems.  Computers  &  Operations  Research,  40, 
558-565. 

[4]  Cordesman,  Anthoney  H,  Gold,  Brian,  &  Ashley,  Hess.  2013.  The  Afghan  War 
in  2013:  Meeting  the  Challenges  of  Transition:  Security  and  the  Afghan  National 
Security  Forces  (CSIS  Reports).  Center  for  Strategic  &  International  Studies. 

[5]  Department  of  the  Army.  2007.  FM  3-04-113  Utility  and  Cargo  Helicopter  Oper¬ 
ations.  Department  of  the  Army. 

[6]  Department  of  the  Army.  2009.  FM  2-01.3  Intelligence  Preparation  of  the  Battle¬ 
field/ Battlespace.  Department  of  the  Army. 

[7]  Department  of  the  Army.  2010a.  FM  3-90.6  Brigade  Combat  Team.  Department 
of  the  Army. 

[8]  Department  of  the  Army.  2010b.  FM  4-90  Brigade  Support  Battalion.  Department 
of  the  Army. 

[9]  Department  of  the  Army.  2011.  ADP  3-0  Unified  Land  Operations.  Department 
of  the  Army. 

[10]  Department  of  the  Army.  2012a.  ADP  4-0  Sustainment.  Department  of  the 
Army. 

[11]  Department  of  the  Army.  2012b.  Cargo  Unmanned  Aircraft  System  (UAS)  Con¬ 
cept  of  Operations. 

[12]  General  Dynamics  Information  Technology.  2010.  Future  Modular  Force  Resupply 
Mission  for  Unmanned  Aircraft  Systems  (UAS).  General  Dynamics  Information 
Technology. 

[13]  Kleywegt,  Anton  J,  Nori,  Vijay  S,  &  Savelsbergh,  Martin  WP.  2002.  The  stochas¬ 
tic  inventory  routing  problem  with  direct  deliveries.  Transportation  Science,  36(1), 
94-118. 


55 


[14]  Kleywegt,  Anton  J,  Nori,  Vijay  S,  &  Savelsbergh,  WP.  2004.  Dynamic  Program¬ 
ming  Approximations  for  a  Stochastic  Inventory  Routing  Problem.  Transportation 
Science ,  38(1),  42-70. 

[15]  Kutner,  Michael  H,  Nachtsheim,  Christoper  J,  &  Neter,  John.  2004.  Applied 
Linear  Regression  Models.  McGraw-Hill/Irwin. 

[16]  Lunday,  Brian  J.,  Sherali,  Hanif  D.,  &  Lunday,  Kevin  E.  2012.  The  Coastal 
Seaspace  Patrol  Sector  Design  and  Allocation  Problem.  Computer  Management 
Science ,  9,  483  -  514. 

[17]  Montgomery,  Douglas  C.  2013.  Design  and  Analysis  of  Experiments.  John  Wiley 
&  Sons,  Inc. 

[18]  Mu,  S,  Fu,  Z,  Lysgaard,  J,  &  Eglese,  R.  2010.  Disruption  Management  of  the 
Vehicle  Routing  Problem  with  Vehicle  Breakdown.  Journal  of  the  Operational 
Research  Society,  62,  742-749. 

[19]  Novoa,  Clara,  &  Storer,  Robert.  2009.  An  Approximate  Dynamic  Programming 
Approach  for  the  Vehicle  Routing  Problem  with  Stochastic  Demands.  European 
Journal  of  Operational  Research,  196,  509-515. 

[20]  Powell,  Warren  B.  2011.  Approximate  Dynamic  Programming:  Solving  the 
Curses  of  Dimensionality.  John  Wiley  &  Sons,  Inc. 

[21]  Puterman,  Martin  L.  1994.  Markov  Decision  Processes:  Discrete  Stochastic 
Dynamic  Programming.  John  Wiley  &  Sons,  Inc. 

[22]  Yousefi,  Arash,  &  Donohue,  George  L.  2004.  Temporal  and  Spatial  Distribution 
of  Airspace  Complexity  for  Air  Traffic  Controller  Workload-Based  Sectorization. 
AIAA  4th  Aviation  Technology,  Integration  and  Operations  (ATIO)  Forum. 


56 


Vita 


Captain  Ian  M.  McCormack  attended  Annapolis  Area  Christian  School,  MD  and 
graduated  in  2001.  He  accomplished  his  undergraduate  studies  at  the  United  States 
Military  Academy  with  a  Bachelor  of  Science  degree  in  Mechanical  Engineering  in 
May  2005.  Ian  was  commissioned  into  the  US  Army  as  a  Second  Lieutenant  of  In¬ 
fantry  in  May  2005. 

Captain  McCormack’s  first  assignment  was  to  the  1st  Cavalry  Division,  2nd 
Brigade  Combat  Team  (HBCT)  at  Fort  Hood,  Texas.  Ian  served  as  a  Platoon  Leader 
in  B/l-8  CAV  during  Operation  Iraqi  Freedom. 

In  August  2008,  Ian  was  assigned  to  the  192nd  Infantry  Brigade  as  an  Assistant 
Brigade  Operations  Officer  at  Fort  Benning,  Georgia.  Ian  then  servered  as  the  Basic 
Training  Company  Commander  of  D/2-54  IN  BN  from  December  2008  -  December 
2009.  Following  command,  Ian  served  as  the  Battalion  Operations  Officer  for  2-54  IN 
BN  from  December  2009  -  January  2012. 

In  January  2012,  Captain  McCormack  attended  the  Maneuver  Captain’s  Career 
Course  at  Fort  Benning,  Georgia.  Following  graduation,  he  was  accepted  into  the 
Functional  Area  49,  Operations  Research/Systems  Analysis. 

In  August  2012,  Ian  entered  the  Air  Force  Institute  of  Technology’s  Graduate 
School  of  Engineering  and  Management  at  Wright-Patterson  AFB,  Ohio.  Upon  grad¬ 
uation,  he  will  be  assigned  to  the  Linked  States  Army  G-8  at  the  Pentagon. 


57 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of 
information,  including  suggestions  for  reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188), 
1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any 
penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  0MB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1.  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE 

27-03-2014  Master's  Thesis 

3.  DATES  COVERED  (From  -  To) 

08-2012  -  03-2014 

4.  TITLE  AND  SUBTITLE 

The  Military  Inventory  Routing  Problem  with  Direct  Delivery 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

McCormack,  Ian  M.,  CPT 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Institute  of  Technology 

Graduate  School  of  Engineering  and  Management  (AFIT/ENS) 

2950  Hobson  Way 

Wright-Patterson  AFB  OH  45433-7765 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

AFIT -EN  S-14-M-20 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

TRADOC  Capability  Manager  for  Unmanned  Aircraft  Systems 

Deputy,  TCM-UAS  Mr.  Glenn  A.  Rizzi 

453  Novosel  Street 

Fort  Rucker,  AF  36362 
glenn.  a.rizzi.civ  @  mail.mil 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

TCM-UAS 

11.  SPONSOR/MONITOR'S  REPORT 

NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Distribution  Statement  A.  Approved  for  Public  Release;Distribution  Unlimited 

13.  SUPPLEMENTARY  NOTES 

This  material  is  declared  a  work  of  the  U.S.  Government  and  is  not  subject  to  copyright  protection  in  the  United  States. 


14.  ABSTRACT 

The  inventory  routing  problem  coordinates  inventory  management  and  transportation  policies  when  implementing  vendor  managed 
inventory  replenishment,  the  business  practice  were  a  vendor  monitors  the  inventory  of  its  customers  and  determines  a  strategy  to 
replenish  each  customer.  The  United  States  Army  uses  vendor  managed  inventory  replenishment  during  combat  situations  to 
manage  resupply.  The  military  variant  of  the  stochastic  inventory  routing  problem  considers  delivery  failure  due  to  hostile  actions. 
We  formulate  a  Markov  decision  process  model  for  the  military  inventory  routing  problem,  with  the  objective  to  determine  an 
optimal  unmanned  tactical  airlift  policy  for  resupplying  brigade  combat  team  elements  in  a  combat  situation  using  cargo  unmanned 
aerial  systems  for  delivery.  Computational  results  are  presented  for  the  military  inventory  routing  problem  with  direct  deliveries. 
Results  indicate  that  unmanned  aerial  systems  are  capable  of  performing  brigade  combat  team  resupply,  given  the  dynamics  of  the 
threat  situation.  An  experimental  design  is  employed  to  determine  the  set  of  factors  important  in  a  more  general  context. 

15.  SUBJECT  TERMS 

logistics,  dynamic  programming,  Markov  decision  process,  deterministic  inventory  routing,  unmanned  aerial  vehicles 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF 

PAGES 

19a.  NAME  OF  RESPONSIBLE  PERSON 

a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

Dr.  Matthew  J.  Robbins,  AFIT/ENS 

U 

U 

U 

UU 

68 

19b.  TELEPHONE  NUMBER  (Include  area  code) 

(937)  255-3636  x4539  matthew.robbins@afit.edu 

Standard  Form  298  (Rev.  8/98) 
Prescribed  by  ANSI  Std.  Z39.18 


